extending dell technologies cloud platform availability

Extending Dell Technologies Cloud Platform Availability for Mission Critical Applications

VMware Cloud Foundation on Dell EMC VxRail Multi-Site Stretched Deployments with Dell EMC PowerMax SRDF/Metro

June 2020

H18367

White Paper

Abstract

This white paper describes a reference architecture used for validating the use of PowerMax enterprise external storage platforms with SRDF/Metro in a VMware Cloud Foundation on a Dell EMC VxRail multisite stretched cluster deployment configuration.

Dell Technologies Solutions

Copyright

2 Extending Dell Technologies Cloud Platform Availability for Mission Critical Applications VMware Cloud Foundation on Dell EMC VxRail Multi-Site Stretched Deployments with Dell EMC PowerMax SRDF/Metro

White Paper

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2020 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA June 2020 White Paper H18367.

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Contents


White Paper

Contents

Executive summary ........................................................................................................................ 4

Introduction ..................................................................................................................................... 5

Solution architecture .................................................................................................................... 11

Configuration ................................................................................................................................. 16

Testing methodology .................................................................................................................... 32

Conclusion ..................................................................................................................................... 38

References ..................................................................................................................................... 39

Executive summary


White Paper

Executive summary

This reference architecture focuses on validating the use of the Dell EMC PowerMax

system with SRDF/Metro in a VMware Cloud Foundation on a Dell EMC VxRail multisite

stretched-cluster deployment configuration. This configuration provides the highest levels

of application availability for customers who are running mission-critical workloads in their

Cloud Foundation on VxRail private cloud.

This white paper shows the value that a Dell Technologies portfolio can provide to

businesses embarking on their digital transformation.

This white paper is for IT professionals who are interested in deploying a private cloud in

their environment and who need validation in leveraging external storage to extend high

availability (HA) services for their mission-critical applications.

Dell Technologies and the authors of this document welcome your feedback on the

solution and the solution documentation. Contact the Dell Technologies Solutions team by

email or provide your comments by completing our documentation survey.

Authors: Jason Marques, Boon Zhang, Xie Chengcai, Yaodong Zhang, Jing Lily,

Krishnamoorthy Karthikeyan

Contributors: Frank Nicolo, Darren Fernandes, Mike Adams, Deepak Vokaliga, Robert

Percy, Aighne Kearney

The following table defines abbreviations that used in this document:

Table 1. List of abbreviations

Term Definition

AVN Application Virtual Network

AZ Availability Zone

CIDR Classless Inter-Domain Routing

HA High Availability

HSRP Hot Standby Router Protocol

IQN iSCSI Qualified name

LCM Lifecycle Management

SDDC Software Defined Data Center

SRDF Symmetrix Remote Data Facility

VCF VMware Cloud Foundation

VI Virtual infrastructure

vMSC VMware vSphere Metro Storage Cluster

vRA vRealize Automation

Document

purpose

Audience

We value your

Feedback

Terminology

mailto:[email protected]?subject=Feedback:%20Extending%20the%20Dell%20Technologies%20Cloud%20Platform%20for%20Mission%20Critical%20Applications%20(H18367)

https://www.surveymonkey.com/r/SolutionsSurveyExt

Introduction


White Paper

Term Definition

vRO vRealize Orchestrator

VTEP Virtual Extensible LAN

VXLAN VXLAN Tunnel End Point

WLD Workload domain

Introduction

Information Technology (IT) departments are under significant pressure to deliver new

applications to market, to innovate with technology to surpass competitors, and to do it

faster and with more choice. At the same time, requirements exist for stricter compliance,

improved security, controlled costs, and increased efficiency. To address these

challenges, the modern data center is trending towards virtualization, converged,

hyperconverged, and software-defined infrastructures, and public and hybrid cloud

solutions.

The VMware vision of the modern data center is one that is software-defined based on a

standardized architecture. The architecture is a fully integrated hardware and software

stack that is simple to manage, monitor, and operate. This architecture for the VMware

software-defined data center (SDDC) empowers companies to run hybrid clouds and

leverage unique capabilities to deliver outcomes that enable efficiency, agility, and

security. The VMware SDDC uses VMware vSphere, VMware vSAN, and VMware NSX to

provide compute, storage, and networking virtualization to the SDDC, and the VMware

vRealize Suite for additional cloud management, self-service, automation, intelligent

operations, and financial transparency.

VMware Cloud Foundation provides integrated cloud infrastructure (vSphere compute,

vSAN storage, NSX networking, and security) and cloud management services (with the

vRealize Suite) to run many types of enterprise applications in both private and public

environments. These applications range from traditional applications that are deployed as

virtual machines (VMs) and VMware Horizon virtual desktops to Kubernetes powered

containerized cloud-native applications Cloud Foundation helps break down the traditional

administrative silos in data centers by merging compute, storage, network provisioning,

and cloud management to facilitate end-to-end support for application deployment. The

Cloud Foundation SDDC Manager component automates the life cycle management of a

complete SDDC on standardized hyperconverged architectures. Customers can deploy

Cloud Foundation on-premises on a broad range of supported hardware or consume it as-

a-service in the public cloud.

Dell Technologies shares VMware’s vision of the modern data center and extends that

vision to the infrastructure. For customers that choose VMware as the primary technology

for modernizing their data center or building a multicloud IT environment, Dell

Technologies offer an automated path to the VMware SDDC with VMware Cloud

Foundation on VxRail.

The following figure shows the components of the VMware Cloud Foundation on VxRail

platform:

Introduction


White Paper

Figure 1. Cloud Foundation on VxRail platform components

As the Dell Technologies Cloud Platform, VMware Cloud Foundation on VxRail delivers a

simple and direct path to the hybrid cloud and Kubernetes at cloud scale with one

complete automated platform. Cloud Foundation on VxRail builds on native VxRail and

Cloud Foundation capabilities, with additional unique integration features that Dell

Technologies and VMware have jointly engineered, to help simplify, streamline, and

automate the operations of your entire SDDC, from before Day 0 through Day 2

operations. This means you get both the hyperconverged infrastructure and cloud

platform software stack in one complete, automated life cycle, turnkey experience.

The platform delivers a set of software-defined services for compute, storage, networking,

security, cloud management, and end-user computing as well as container-based cloud-

native platform services, in both private and public environments. This makes it an ideal

operational hub for your hybrid cloud. To accelerate an organization’s move to adopt

containers and a hybrid cloud operating model, Dell Technologies offers unique

integration between Cloud Foundation and VxRail. The integration supports VM-based

and container-based workloads at the same time on Dell EMC PowerEdge servers, with

support for optionally incorporating Dell EMC external storage systems across multiple

cloud environments.

VMware Cloud Foundation on VxRail makes operating the data center fundamentally

simpler by bringing the ease and automation of the public cloud in-house. This is achieved

by deploying a standardized and validated network-flexible architecture with integrated

full-stack life cycle automation for the entire cloud infrastructure stack, including hardware.

This level of deep integration with Cloud Foundation is what gives customers a truly

unique turnkey hybrid cloud experience.

Introduction


White Paper

Further, because security is one of the top priorities for the data center, Cloud Foundation

on VxRail includes integrated end-to-end security at every level of the infrastructure --

from compute and management security to microsegmentation at the networking layer, to

encryption at the storage layer, to security down into the hardware.

These benefits are also available with single vendor support through Dell Technologies

along with choices for infrastructure delivery and flexible financial consumption options.

Organizations that are looking to extend their private cloud to the public cloud can

consume VMware Cloud Foundation as-a-service through a broad range of cloud service

provider partners, including VMware Cloud on AWS, Microsoft Azure, Google Cloud, IBM

Cloud, Rackspace, and others. This enables a true hybrid cloud that is based on a

common and compatible platform that stretches from on-premises to off-premises for your

VMs and containerized modern applications.

Today, Dell Technologies customers have a lot to consider when choosing the storage

that they need to run their workloads. Their storage requirements must scale easily and

meet the I/O demands of their applications.

Organizations with large databases, such as those used for big data, machine learning

(ML), and AI initiatives, have storage-intensive requirements. Storage must be cost-

effective and scale across multiple tiers.

Data protection (backup), disaster recovery, and multisite high availability are other

factors in a storage footprint. The flexibility to support different storage types across

different platforms is also important. This flexibility helps organizations contribute a

consistency that is nondisruptive to their hybrid cloud and multicloud strategy. VMware

Cloud Foundation on VxRail offers flexible storage connectivity options that cater to these

different use cases to assist customers in their transition to a software-defined platform

and operating model.

Customers commonly ask whether they can use external storage in Cloud Foundation on

VxRail deployments. The answer is Yes. This helps customers ease into the transition to

a software-defined architecture from an operational perspective. It also helps customers

leverage the investments in their existing infrastructure for the many different workloads

that might still require external storage services.

External storage

with Cloud

Foundation on

VxRail

Introduction


White Paper

External storage and Cloud Foundation have two important use cases: principal storage

and supplemental storage.

• Principal storage—SDDC Manager provisions a workload domain that uses vSAN,

NFS, or Fibre Channel (FC) storage for a workload domain cluster’s principal

storage (the initial shared storage that is used to create a cluster.

By default, VCF uses vSAN storage as the principal storage for a cluster. The

option to use NFS and FC-connected external storage is also available. This option

enables administrators to create a workload domain cluster whose principal storage

can be an NFS datastore or an FC-based VMFS datastore instead of vSAN. SDDC

Manager provisions a workload domain (WLD) cluster by attaching a previously

created NFS file system or FC LUN to it instead of using and configuring a vSAN

storage for that cluster. In both cases, the administrator manually creates the

external storage ahead of time. The network (IP or FC) must also be configured or

zoned ahead of time so that the nodes that will be used for the workload cluster

have access to the network. The mounting and formatting of the storage is

automated as part of the SDDC Manager ‘create WLD’ cluster workflow.

Importantly, Cloud Foundation does not provide extensible LCM capabilities for the

external storage systems. That is still a separately managed process.

• Supplemental storage—This involves manually mounting previously provisioned

external NFS, iSCSI, vVols, or FC storage to a Cloud Foundation workload domain

cluster that is using vSAN as the principal storage. Supporting external storage for

these workload domain clusters is comparable to the experience of administrators

using standard vSphere clusters who want to attach secondary datastores to those

clusters. For Cloud Foundation, this mounting process is performed manually using

standard vCenter operations procedures. Cloud Foundation does not take

ownership of the life cycle management, storage provisioning, or network (IP or FC)

configuration or zoning for the external storage systems. Instead, administrators

use the native storage system and network management tools for those tasks.

At the time of writing, Cloud Foundation on VxRail supports supplemental storage use

cases only.

The Dell EMC PowerMax is the first Dell EMC hardware platform that uses an end-to-end

Non-Volatile Memory Express (NVMe) architecture for customer data. NVMe is a set of

standards that define a PCI Express (PCIe) interface used to efficiently access data

storage volumes based on Non-Volatile Memory (NVM) media, which includes modern

NAND-based flash along with higher-performing Storage Class Memory (SCM) media

technologies. The NVMe-based PowerMax array fully unlocks the bandwidth, IOPS, and

latency performance benefits that NVM media and multi-core CPUs offer to host-based

applications—benefits that are unattainable using the previous generation of all-flash

storage arrays.

The following figure shows the PowerMax 2000 and PowerMax 8000 models:

Use cases

PowerMax family

Introduction


White Paper

Figure 2. PowerMax product family

The primary benefits that PowerMax platforms offer to customers include:

• A powerful end-to-end NVMe storage architecture that delivers:

▪ Up to 15M IOPS, 350 GBps throughput (187K IOPS per rack unit)

▪ Industry-standard NVMe-based flash and SCM drives

▪ Native NVMe Drive Array Enclosures (DAEs)

• The ability to use current Gen 6 FC (FC-NVMe) infrastructures and future-ready

Gen 7 FC infrastructures

• Enterprise levels of reliability designed for 99.9999 percent availability in a single

array

• Investment protection with the Dell Technologies Future Proof Loyalty Program

• Massive workload consolidation by supporting Open Systems (FC, FC-NVMe,

iSCSI), Mainframe, IBM i, containers, and file storage on the same array,

simplifying management and significantly lowering the total cost of ownership

(TCO)

• Storage provisioning operations in under 30 seconds with Dell EMC Unisphere for

PowerMax

• CloudIQ, a simple app to track storage health, report on historical trends, plan for

future growth, and proactively discover and remediate issues from any browser or

mobile device

• An integrated real-time ML engine for automatic data placement

▪ Automated I/O recognition and data placement across flash and SCM media to

maximize performance with no management overhead

Introduction


White Paper

▪ Elimination of high-performance silos and consolidation of all mission-critical

workloads and secondary applications

• Global inline deduplication and enhanced compression with virtually no

performance impact

▪ Data reduction that works with all data services

▪ Granular control that can be turned on or off by application (storage group)

• Unprecedented storage security and protection

▪ Controller-based Data at Rest Encryption (D@RE), FIPS 140-2 validated,

secure snapshots, role-based authentication, and tamper-proof audit logs

• Industry-leading high availability

▪ Non-disruptive hardware and software upgrades

▪ Nondisruptive migrations

PowerMaxOS 5978 and Solutions Enabler/Unisphere for PowerMax 9.1, which were

released in Q3 2019, introduced support for SRDF/Metro Online Device Expansion (ODE)

and a new Unisphere interface for adding and removing SRDF/Metro devices based on

the existing storage group add/remove device workflow. From Unisphere for PowerMax

and Solutions Enabler 9.1on, ODE support was expanded to include devices taking part in

SRDF/Metro (Active) sessions. This new functionality is based on modifications to our

existing Geometry Compatibility Mode (GCM) functionality for host visibility of devices.

Unisphere 9.1 also provides new ease-of-use functionality by automating the addition of

devices to a storage group, including corresponding SRDF paired devices for single-hop,

concurrent, and cascaded SRDF configurations.

With SRDF/Metro, the SRDF secondary device is read/write accessible to the host and

takes on the external identity of the primary device (its geometry, device WWN, and so

on). By providing this external identity on the secondary device, both the primary and

secondary devices appear as a single virtual device across the two SRDF paired arrays

for presentation to a single host or host cluster. With both devices accessible, the host (or

hosts in the case of a cluster) can read and write to both primary and secondary devices

with SRDF/Metro, ensuring that each copy remains current and consistent and addressing

any write conflicts that might occur between the paired SRDF devices. A single PowerMax

All Flash array can simultaneously support multiple SRDF groups that are configured for

SRDF/Metro operations and multiple SRDF groups that are configured for non-

SRDF/Metro operations.

The key differences between SRDF/Metro and standard synchronous and asynchronous

SRDF modes are:

• All SRDF device pairs that are in the same SRDF group and are configured for

SRDF/Metro must be managed together for all supported operations, with two

exceptions:

If all the SRDF device pairs are not ready (NR) on the link, the user may:

▪ Perform a createpair operation to add devices to the SRDF group, provided

the new SRDF device pairs are created not ready (NR) on the link.

SRDF/Metro

Solution architecture


White Paper

▪ Perform a deletepair operation on all or a subset of the SRDF devices in the

SRDF group.

• An SRDF device pair taking part in an SRDF/Metro configuration can be brought to

the following state:

▪ Both sides of the SRDF device pair appear to the host or hosts as the same

device.

▪ Both sides of the SRDF device pair are accessible to the host or hosts.


This section describes the active/active solution design that we implemented by using a

Cloud Foundation on a VxRail multisite stretched-cluster deployment configuration with

PowerMax SRDF/Metro. This design provides enhanced levels of flexibility and availability

that extend the core capabilities of the cloud platform. The VMware Cloud Foundation on

VxRail solution natively supports a stretched-cluster configuration for the management

domain and a VI workload domain between two availability zones by using vSAN

stretched clusters. A PowerMax SRDF/Metro with vMSC configuration is added to protect

VI workload domain workloads by using supplementary storage for the workloads that are

running on them.

Two types of vMSC configuration are verified with stretched Cloud Foundation on VxRail:

uniform and nonuniform.

• Uniform host access configuration—vSphere hosts from both sites are all

connected to a storage node in the storage cluster across all sites. Paths presented

to vSphere hosts are stretched across a distance.

• Nonuniform host access configuration—vSphere hosts at each site are

connected only to storage nodes at the same site. Paths presented to vSphere

hosts from storage nodes are limited to the local site.

The following figure shows the topology of the Cloud Foundation on VxRail uniform

stretched-cluster configuration with PowerMax SRDF/Metro.



White Paper

Figure 3. Cloud Foundation on VxRail uniform stretched-cluster configuration with PowerMax SRDF/Metro

The following figure shows the topology of the Cloud Foundation on VxRail nonuniform

stretched-cluster configuration with PowerMax SRDF/Metro:



White Paper

Figure 4. Cloud Foundation on VxRail nonuniform stretched-cluster configuration with

PowerMax SRDF/Metro

In total, we used fourteen VxRail E460F nodes running VxRail version 4.7.410 code on

each node, which aligns with Cloud Foundation 3.9.1 requirements. For a complete bill of

materials, see VMware Cloud Foundation 3.9.1 on Dell EMC VxRail Release Notes.

The following table shows the hardware components that we used in this configuration.

For this deployment, we deployed the VMware Cloud Builder utility on the VCF

management domain cluster. If required, you can deploy this utility on separate

infrastructure. We used a separate PowerEdge R630 server to host the vSAN witness

components for the stretched clusters.

Table 2. Hardware resources

Hardware Quantity Purpose Operating system/Firmware

VxRail E460F server

8 Stretched VCF management domain, VCF Cloud Builder VM, and NSX DHCP VM

ESXi6.7 U3b

VxRail E460F server

6 Stretched VI workload domain connected to PowerMax SRDF/Metro devices

ESXi6.7 U3b

Dell EMC PowerEdge R630 server

1 Stand-alone server used as witness host for VCF vSAN stretched cluster

ESXi6.7 U3b

Hardware

components

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9.1/rn/vmware-cloud-foundation-on-dell-emc-vxrail-16-release-notes.html



White Paper

Hardware Quantity Purpose Operating system/Firmware

PowerMax 8000 Array

1 External storage for VCF VI workload domain. R1 of SRDF configuration

PowerMaxOS 5978

PowerMax 2000 Array

1 External storage for VCF VI workload domain. R2 of SRDF configuration

PowerMaxOS 5978

Cisco Nexus 9372PX Switch

2 L3 switches are used for VCF management, vMotion, vSAN, NSX VxLAN, vRealize Network

7.0(3)I5(2)

The following table shows the software components that we used in this configuration:

Table 3. Software resources

Software Version

VMware Cloud Foundation 3.9.1

VMware vSphere ESXi6.7 U3b

Dell EMC VxRail 4.7.410

Dell EMC PowerPath/VE 7.0

Dell EMC Unisphere for PowerMax 9.2.0.1194

Dell EMC vRO plug-in for PowerMax 1.2.0.397

For information about firmware and software versions, see Dell EMC VxRail Appliance

Release Notes, Version 4.7.x.

The following table shows the infrastructure VMs that we used in this configuration:

Table 4. Infrastructure VMs

Infrastructure VM Purpose

DHCP VM VxLAN - DHCP Network

vCenter Temporary vCenter to add the ESXi server and configure NTP and hostname

VCF Cloud Builder VM Build up the VCF management domain

vSAN Witness Witness VM for stretched VCF cluster from VMware-VirtualSAN-Witness-201912001-15160138.ova

Unisphere VM A Windows 2012R2 VM runs the Dell EMC Unisphere application

Dell EMC PowerPath Management Appliance

Provides license automation to support Dell EMC PowerPath/VE. The PPMA version is 2.6

vCenter Two in the management cluster: the primary vCenter for the management domain and the vCenter for the workload domain

Software

components

Infrastructure

virtual machines

https://dl.dell.com/content/docu91467_VxRail_Appliance_Software_4.7.x_Release_Notes.pdf?language=en_US&source=Coveo

https://dl.dell.com/content/docu91467_VxRail_Appliance_Software_4.7.x_Release_Notes.pdf?language=en_US&source=Coveo



White Paper

Infrastructure VM Purpose

Jumpbox Windows VM provides RDP service to access the VCF cluster

For Ethernet/iSCSI:

• Each node is connected to two 10G uplinks. A pair of Cisco Nexus 9372PX

switches is used as top-of-rack (ToR) switches.

• Uplink connectivity to the core network through the ToR switches.

• Intel X520 dual-port 10Gb NIC is used for iSCSI connections to PowerMax. Each

VxRail workload domain node has one network daughter card (NDC).

For FC:

• Emulex LPe16002 dual-port 16Gb FC adapter is used for the FC connection to

PowerMax. Each VxRail workload domain node has one FC HBA card.

The following figure shows the physical connectivity between the servers, arrays, and

switches:

Figure 5. Physical connectivity between servers, arrays, and switches

Connectivity

Configuration


White Paper

Configuration

The building block of this solution uses VMware Cloud Foundation 3.9.1 on VxRail

4.7.410 with PowerMax 8000/2000. The following sections of this document describe the

key configuration steps.

Multiple VLANs are designed to isolate traffic across AZ1 and AZ2. All networking is

performed at layer 2. No layer 3 routing is required on the network for AZ1 hosts to reach

AZ2 hosts.

The following figure shows the network topology:

Figure 6. Network topology

L2 network overview

The management domain network configuration is recorded in the vcf-vxrail-deployment-

parameter.xlsx spreadsheet. This configuration is used for deploying the management

domain on AZ1 using the VMware CloudBuilder utility.

• The port group names vCenter Server Network, vSphere vMotion, and vSAN are

VxRail default names. These names are not editable in the VCF deployment

parameter spreadsheet.

• The port group name VXLAN (VTEP) is the default name for NSX-v deployments in

VCF 3.9.1. Where “n/a” is displayed for CIDR and gateway, these values are not

relevant for the VCF deployment. For vMotion and vSAN, CIDR is set during the

VxRail cluster deployment. The VXLAN (VTEP) CIDR is set by DHCP.

The following table shows the management domain network configuration details:

Networking

Configuration


White Paper

Table 5. Management domain networks

VLAN number

Port group name

CIDR notation

Gateway MTU

2312 vCenter Server Network

10.226.131.64/26

10.226.131.65 1500

1602 vSphere vMotion n/a n/a n/a

1601 vSAN n/a n/a n/a

1603 VXLAN (VTEP) - DHCP Network

n/a n/a 9000

2313 nsxv-uplink01 10.226.131.128/28

10.226.131.129 9000

2314 nsxv-uplink02 10.226.131.144/28

10.226.131.145 9000

L2 port setting

Ports for all hosts are configured in the management cluster, as shown in the following

figure:

Figure 7. Switch port interface setting

VLAN setting on AZ1 and AZ2

The default gateways for the management network and NSX-v uplink networks are

configured on the network core. HSRP is used for redundancy, as shown in the following

figure:

Figure 8. VLAN interface setting

Configuration


White Paper

• The VxRail ToR switches are connected to the lab core switches. The lab network

is entirely layer 2, so all default gateways are configured on the core switches. No

default gateways are configured on the VxRail ToR switches because both

management and workload domain clusters are stretched using layer 2.

• All VLANs are stretched from AZ1 to AZ2 and subnets. The default gateways are

the same.

vSAN Witness

Two vSAN witness appliances are deployed to the stand-alone ESXi host, one each for

the management and workload domain clusters. Layer 2 VLANs are stretched to the “third

site” to facilitate the layer 2 stretch deployment methodology. vSAN witness appliances

are deployed onto the management VLAN with an additional interface on the vSAN VLAN.

Configuring VCF stretched-cluster deployments requires first stretching the management

domain cluster and then stretching workload domain clusters, as described in the

following procedure.

Management domain cluster:

1. Stretch VLANs for management, vSAN, and vMotion to AZ2.

2. Use the SoS command within SDDC CLI to prepare the cluster to be stretched.

3. Power on AZ2 management domain hosts and perform cluster expansion in

SDDC Manager.

4. Deploy the vSAN witness appliance to an ESXi host in a third location and add

the appliance to the domain being stretched.

5. Use the SoS command in the SDDC CLI to complete the L2 stretch operation.

6. Monitor vSAN health to ensure that the stretch operation is successful.

VI workload domain clusters:

• Repeat the preceding steps for each workload domain cluster to be stretched.

We used the same process to stretch management and workload domains across two

availability zones. To configure layer 2, we followed the process that is described in the

VMware document Stretch a Cluster for NSX-V in VMware Cloud Foundation 3.9.1.

First deploy the Cloud Foundation stretched management domain cluster on VxRail, and

then deploy the Cloud Foundation stretched workload domain cluster on VxRail. After a

successful deployment, add the workload domain by using the wizard that is available in

the SDDC Manager UI.

The following figure shows an example:

Stretched-cluster

configuration in

Cloud

Foundation

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.vxrail.admin.doc/GUID-49E5F547-760D-49B2-BAC2-BEF6EEAD604E.html

Configuration


White Paper

Figure 9. SDDC Manager UI: W Dashboard view of stretched VxRail management and VI workload domains

Deploying vRA with embedded vRO in Cloud Foundation involves:

• Deploying the vRealize Suite Lifecycle Manager virtual appliance onto the

management domain.

• Deploying vRA using the vRSLCM onto the management domain.

• Enabling embedded vRO within the vRA deployment.

The following sections of the document describe these procedures.

Deploy vRealize Suite Lifecycle Manager using SDDC Manager

In the SDDC Manager UI:

1. From the VMware repository, download the vRealize LCM bundle.

2. Create all required DNS records.

3. Use the wizard that is available in SDDC Manager to deploy vRSLCM onto the

management domain.

The following figure shows the vRealize Suite Lifecycle Manager UI:

Deploying

vRealize

Automation and

vRealize

Orchestrator

Configuration


White Paper

Figure 10. vRealize Suite Lifecycle Manager

Deploy vRealize Automation

In the SDDC Manager UI:

1. Add the vRA license key.

2. Create all the DNS records that are required for vRA components.

3. Download the vRA bundle from the VMware repository.

4. Create a multi-SAN SSL certificate request within the SDDC Manager CLI and

sign it using lab certificate authority.

5. Create Microsoft SQL server and configure it for vRA.

6. Create a Windows OVA template for vRA IaaS components.

7. Deploy vRA from within the vRealize Suite menu of the SDDC Manager UI.

Notes:

vRA is deployed to the SDDC management cluster, onto an NSX Application Virtual Network

(AVN) which you specify during the Cloud Foundation deployment.

After successful deployment of vRA, add the workload domain by using the wizard that is

available in the SDDC Manager UI. The following figure shows an example:

Configuration


White Paper

Figure 11. SDDC Manager UI: Deploying vRealize Automation

For more information about the preceding steps, see the VMware document Deploy

vRealize Automation in Cloud Foundation.

Enable vRealize Orchestrator

To automatically create SRDF/Metro on PowerMax, you must enable embedded vRO on

each vRA node:

1. SSH into vRA node with the root account.

2. Run chkconfig vco-configurator to verify the service.

3. Run chkconfig vco-configurator on to enable vRO at boot on vRA node.

4. Run service vco-configurator status to check the service status.

The following figure shows the vRealize Orchestrator UI:

Figure 12. vRealize Orchestrator UI

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.admin.doc_39/GUID-1112AEAD-9A6A-41F7-8C83-27F279CC8517.html

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.admin.doc_39/GUID-1112AEAD-9A6A-41F7-8C83-27F279CC8517.html

Configuration


White Paper

For more information about the process, see the VMware document Start the vRealize

Orchestrator Configurator Service in Cloud Foundation.

The PowerMax SRDF/Metro feature enables an administrator to host applications in HA

environments. When configured using SRDF/Metro, hosts, clusters, and applications can

perform read/write operations on both source and target arrays. The feature functions like

SRDF/S mode and maintains a real-time copy at arrays that are located within 200 KMs,

depending on application workload, network latency, and block size.

To create SRDF/Metro pairs:

1. Create volumes on both source and target arrays as needed.

2. Add the volumes to storage groups on each side.

3. Create SRDF/Metro pairs between the source and target array (see Create

SRDF/Metro pairs with vRO plug-in for PowerMax).

4. Wait until the synchronization completes and the arrays are in an active/active

pair state.

5. Perform storage provisioning tasks on the source and target arrays (see Create

SRDF/Metro pairs with vRO plug-in for PowerMax).

Note: SRDF/Metro offers customers two types of witness deployment models: a physical array

witness and a software witness. We configured this validated SRDF/Metro environment using the

physical array witness.

Create SRDF/Metro pairs with vRO plug-in for PowerMax

The vRO plugin for PowerMax enables the administrator to provide automation

capabilities for Dell EMC VMAX All Flash and PowerMax storage arrays within vRO. The

plug-in helps automate the operations that are required for provisioning and protection

with prescheduled and customized workflows.

1. The first time the vRO plug-in for PowerMax is used, add the Storage

Management Server using Dell EMC Unisphere for PowerMax.

Run the “Add Storage Management Server” workflow, and then run the “Register

Storage System” workflow, as shown in the following figure:

PowerMax

SRDF/Metro

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.admin.doc_39/GUID-09C3E9B1-9739-4A5B-A270-A4654C9499FB.html

https://docs.vmware.com/en/VMware-Cloud-Foundation/3.9/com.vmware.vcf.admin.doc_39/GUID-09C3E9B1-9739-4A5B-A270-A4654C9499FB.html

Configuration


White Paper

Figure 13. Adding and registering Dell EMC Unisphere for PowerMax in vRO

2. Create the host on the source and target PowerMax, as shown in the following

figure. Run the “Create Host” workflow.

For FC:

▪ If the FC zoning is complete, the initiator WWN has already logged into

PowerMax. Select the WWN in Pick initiator WWNs from available ones on

array. When you click + and enter part of the WWN, the full WWN appears

automatically.

▪ If the FC zoning is not complete, provide the initiator WWNs manually.

Configuration


White Paper

Figure 14. Create host for FC

For iSCSI:

▪ Provide the IQN manually, as shown in the following figure:

Configuration


White Paper

Figure 15. Create host for iSCSI

3. Configure auto storage provisioning on the source PowerMax. Run the “Provision

Volumes to Host Group” workflow.

In the Host Group tab, enter the host group name and select the hosts that you

created in the preceding step. Select Fiber or iSCSI:

▪ For a uniform vMSC configuration, select AZ1 and AZ2 hosts into the host

group, as shown in the following figure:

Configuration


White Paper

Figure 16. Provision volumes - host group for uniform

▪ For a nonuniform vMSC configuration, select AZ1 hosts into the host group for

PowerMax R1 and select AZ2 hosts into the host group for PowerMax R2, as

shown in the following figure:

Configuration


White Paper

Figure 17. Provision volumes - host group for nonuniform

4. On the Masking View tab, provide the masking view name, as shown in the

following figure:

Figure 18. Provision volumes - masking view

5. In the Port Group tab:

For FC, enter the target port WWN. The PowerMax FA port is displayed

automatically, as shown in the following figure:

Configuration


White Paper

Figure 19. Provision volumes – port group for FC

▪ For iSCSI, enter the target port IQN. The PowerMax target port IQN is

displayed, as shown in the following figure:

Figure 20. Provisioning volumes: Host group for iSCSI

6. In the Storage Group tab, as shown in the following figure, select New storage

group and enter the storage group name.

Configuration


White Paper

Figure 21. Provision volumes - storage group

7. On the Volumes tab, as shown in the following figure, select New volumes and

provide the required details about the volumes.

Figure 22. Provision volumes - volumes

8. Click RUN and wait for the workflow to complete successfully, as shown in the

following figure:

Configuration


White Paper

Figure 23. Provision volumes – complete status

9. Check the PowerMax devices on the VxRail nodes of the VI workload domain

cluster.

Only VxRail nodes in AZ1 can see the PowerMax devices, as shown in the

following figure:

Figure 24. PowerMax devices shown on VCF AZ1 server

10. Create PowerMax SRDF/Metro:

a. Run the “Create Storage Group SRDF Protection” workflow.

b. Enter the source and target array information.

c. In SRDF replication mode, select Active.

The workflow creates the volumes on the target array and establishes the

SRDF/Metro pairs, as shown in the following figure:

Configuration


White Paper

Figure 25. Create storage group SRDF protection - general

Figure 26. Create storage group SRDF protection - advanced options

11. Configure auto storage provisioning on the remote PowerMax.

a. Run the “Provision Volumes to Host” workflow.

Testing methodology


White Paper

b. On the Storage Group tab, enter the storage group name that you created

in the preceding step in “Existing storage group name.”

If it is a uniform configuration, the PowerMax device paths are doubled, as

shown in the following figure:

Figure 27. PowerMax device paths with vMSC uniform configuration

If it is a nonuniform configuration, the AZ2 servers start to see the PowerMax

devices, as shown in the following figure:

Figure 28. PowerMax device paths with vMSC nonuniform configuration

Testing methodology

We implemented four test scenarios for our validation.

For FC:

• VMware VCF on VxRail with PowerMax SRDF/Metro FC uniform configuration plus

NMP

• VMware VCF on VxRail with PowerMax SRDF/Metro FC nonuniform configuration

plus Dell EMC PowerPath/VE 7.0

Test scenarios

Testing methodology


White Paper

For iSCSI:

• VMware VCF on VxRail with PowerMax SRDF/Metro iSCSI uniform configuration

plus Dell EMC PowerPath/VE 7.0

• VMware VCF on VxRail with PowerMax SRDF/Metro iSCSI nonuniform

configuration plus NMP

We tested the following configurations:

• VM operating system type: Windows 2012/Windows 2016,

Red Hat RHEL6/Red Hat RHEL7

• The I/O tool fio run on each VM

Functional verification testing

This testing addresses the basic operations that are performed when PowerMax is used

as supplementary storage with VMware VCF on VxRail. The following table describes the

test cases and the expected results:

Table 6. Functional verification testing

Number Test case Expected result

1 PowerMax SRDF/Metro LUN provision to servers of workload domain

PowerMax SRDF/Metro LUN can be used for supplementary storage to VI workload domain

2 VMFS5\6 creation on PowerMax SRDF/Metro LUN over 2 TB

Both VMFS5 and VMFS6 datastores are verified on PowerMax SRDF/Metro devices. They all can be read and written to in the VxRail stretched cluster

3 VM deployment, Power On\Off & Suspend\Resume on PowerMax SRDF/Metro LUN

VM can be deployed on PowerMax SRDF/Metro devices in VI workload domain and all VM operations work properly

4 VM snapshot create and revert and delete on PowerMax SRDF/Metro LUN

The operation on VM snapshot works well on PowerMax SRDF/Metro device on VI workload domain

5 VM clone on PowerMax SRDF/Metro LUN Clone VMs which are running I/O on PowerMax. SRDF/Metro device works correctly

6 VM (newly created and workload domain-initiated NSX appliance) can perform storage migration bidirectionally between PowerMax SRDF/Metro LUN

VM storage migration works on both the primary storage and supplementary storage of VI the workload domain

Test

configurations

Test case details

Testing methodology


White Paper


7 LUN hot add/remove with PowerMax SRDF/Metro LUN to workload domain cluster node

PowerMax SRDF/Metro devices can be hot added and removed in VI workload domain

Note: The SRDF group is

required to be suspended before

you add or remove a device on

the existing SRDF group. There

is no I/O interruption on

PowerMax SRDF/Metro uniform

configuration. The I/O on AZ2 is

interrupted on PowerMax

SRDF/Metro nonuniform

configuration.

8 Online device expansion on PowerMax SRDF/Metro LUN in workload domain

Perform online device expansion on any PowerMax site. All sites of PowerMax/SRDF devices can be updated simultaneously.

9 VM migration in workload domain cluster on PowerMax SRDF/Metro

VM migration within the datastore which is supplementary to the VI workload domain,

High availability testing

HA testing helps validate the capability of the solution to avoid a single point of failure

from the hardware component port level up to the IDC site level. The following table

describes the HA testing that we performed:

Table 7. High availability testing


1 Maintenance work for management domain level HA protection

The VCF on VxRail stretch cluster keeps working without any interruption.

2 Host side HBA port failure triggered I/O path failover

Host FC/iSCSI connection is partly failed. There is no I/O interruption in such a condition.

3 Storage-side service processor port failure triggered I/O path failover

One PowerMax director is in failure. The I/O does not interrupt because FC zoning and storage provision were also performed on different PowerMax directors to provide a failover path.

4 Storage firmware NDU No I/O interruption during the storage firmware upgrade.

Testing methodology


White Paper


5 VCF Stretch Cluster-WLD AZ1 host unplanned failure triggered HA protection

The powered-on VMs on the AZ1 host can be restarted on the AZ2 host.

The powered-on VMs on AZ2 keep running with no I/O interruption.

6 Network failure between WLD AZ1 hosts and AZ2 hosts

The AZ1 and AZ2 network isolation does not impact the I/O running on a VI workload domain with PowerMax SRDF/Metro.

7 PowerMax SRDF/Metro: unplanned failure on one of the PowerMax nodes (R1 or R2)

While one PowerMax is dead:

- For vMSC uniform configuration: no I/O interruption while one PowerMax is down.

- For vMSC nonuniform configuration: the VMs on the surviving PowerMax continue to run without interruption, the VMs on the dead PowerMax restart on the surviving side with VMware HA PDL enabled after the PowerMax is online, all PowerMax SRDF/Metro devices path are active, and VMs can be migrated between AZ1 and AZ2.

8 PowerMax SRDF/Metro feature test for interlink break triggered VM level HA protection

For uniform vMSC configuration: no I/O interruption on both AZ1 and AZ2.

For nonuniform vMSC configuration: VMs on AZ2 that connect to PowerMax R2 are restarted on AZ1 and connected to PowerMax R1 when HA PDL is enabled on the VI workload domain.

Reliability testing

In general, reliability testing validates whether the components and the whole system are

reliable enough with a certain level of stress running on them.

Testing methodology


White Paper

Table 8. Reliability testing


1 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to the workload domain

All VM power-on operations can be successfully run simultaneously.

2 Scale testing for long-time stress I/O running in all VMs

Continually running I/O for more than 24 hours without error

Table 9. Test results for FC

Number Test case FC uniform with NMP

FC nonuniform with PPVE7.0


Pass Pass


Pass Pass

3 VM deployment, Power On\Off, and Suspend\Resume on PowerMax SRDF/Metro LUN

Pass Pass


Pass Pass

5 VM clone on PowerMax SRDF/Metro LUN

Pass Pass

6 VM (newly created and workload domain initiated NSX appliance) can do storage migration bidirectionally between PowerMax SRDF/Metro LUN

Pass Pass


Pass Pass


Pass Pass


Pass Pass

10 Host-side HBA port failure triggered I/O path failover

Pass Pass

11 Storage-side SP port failure triggered I/O path failover

Pass. Pass

12 Storage firmware NDU Pass Pass

Testing methodology


White Paper

Number Test case FC uniform with NMP

FC nonuniform with PPVE7.0


Pass Pass

14 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to workload domain

Pass Pass

15 Scale testing for long time stress I/O running in all VMs

Pass Pass

16 VCF stretch cluster-WLD AZ1 host unplanned failure triggered HA protection

Pass Pass


Pass Pass


Pass Pass

19 PowerMax SRDF/Metro: unplanned failure on one of PowerMax Node (R1 or R2)

Pass Pass

Table 10. Test results for iSCSI

Number Test case iSCSI uniform with PPVE7.0

iSCSI nonuniform with NMP


Pass Pass


Pass Pass

3 VM deployment, Power On/Off and Suspend/Resume on PowerMax SRDF/Metro LUN

Pass Pass


Pass Pass

5 VM clone on PowerMax SRDF/Metro LUN Pass Pass

6 VM (newly created and workload domain initiated NSX appliance) can perform storage migration bidirectionally between PowerMax SRDF/Metro LUN

Pass Pass


Pass Pass


Pass Pass

Conclusion


White Paper

Number Test case iSCSI uniform with PPVE7.0

iSCSI nonuniform with NMP


Pass Pass

10 Host-side HBA port failure triggered I/O path failover

Pass Pass

11 Storage-side SP port failure triggered /IO path failover

Pass Pass

12 Storage firmware NDU Pass Pass


Pass Pass

14 Scale testing for more than 100 VMs operating simultaneously on PowerMax SRDF/Metro LUNs attached to workload domain

Pass Pass

15 Scale testing for long-time stress I/O running in all VMs

Pass Pass

16 VCF stretch cluster-WLD AZ1 host unplanned failure triggered HA protection

Pass Pass


Pass Pass


Pass Pass

19 PowerMax SRDF/Metro: unplanned failure on one of PowerMax Node (R1 or R2)

Pass Pass

Conclusion

This white paper describes how Dell EMC engineers integrated VMware Cloud

Foundation on VxRail with PowerMax SRDF/Metro and the design configuration steps

that they took to automatically provision PowerMax storage by using the PowerMax vRO

plug-in. The paper validates that the Cloud Foundation on VxRail solution functions as

expected in both a PowerMax uniform vMSC configuration and a nonuniform vMSC

configuration by passing all the designed test cases. This reference architecture validation

demonstrates the power of the Dell Technologies portfolio to provide customers with

modern cloud infrastructure technologies that deliver the highest levels of application

availability for business-critical and mission-critical applications running in their private

clouds.

References


White Paper

References

The following documents on the Dell Technologies online support website provide

additional information. Access to these documents depends on your login credentials. If

you do not have access to a document, contact your Dell Technologies representative:

• Dell EMC PowerMax and VMAX All Flash: SRDF/Metro Overview and Best

Practices

• Dell EMC Unisphere for PowerMax Installation Guide

• vRO Plug-in for Dell EMC PowerMax Installation Guide

• BEST PRACTICES FOR USING Dell EMC SRDF/METRO IN A VMWARE

VSPHERE METRO STORAGE CLUSER

• VMware vSphere Metro Storage Cluster (vMSC) with Dell EMC PowerMax and

VMAX SRDF/Metro (Partner Verified and Supported) (2134684)

• VMware Cloud Foundation 3.9.1 on Dell EMC VxRail Release Notes

• VxRail-Appliance-Software-4.7.x-Release-Notes

• Stretch a Cluster for NSX-V in VMware Cloud Foundation 3.9.1

• Deploy vRealize Automation in Cloud Foundation

• Start the vRealize Orchestrator Configurator Service in Cloud Foundation

https://support.dell.com/

extending dell technologies cloud platform availability

Documents