to see "hpe 3par vmware esx/esxi implementation guide"

HPE 3PAR VMware ESX/ESXiImplementation Guide

AbstractThis implementation guide provides information for establishing communication between an HPE 3PAR StoreServ Storage anda VMware ESX/ESXi host. General information is provided on the basic procedures required to allocate storage on the 3PARStoreServ Storage that is accessed by the host.

Part Number: QL226-99141Published: April 2016

© Copyright 2012, 2016 Hewlett Packard Enterprise Development LP

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and servicesare set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constitutingan additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistent with FAR12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensedto the U.S. Government under vendor's standard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is notresponsible for information outside the Hewlett Packard Enterprise website.

Acknowledgments

Red Hat® is a registered trademark of Red Hat, Inc. in the United States and other countries.

VMware®, VMware® vCenter™, and VMware vSphere® are registered trademarks or trademarks of VMware, Inc. in the United States and/or

other jurisdictions.

Windows® and Windows Server® are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other

countries.

Contents1 Introduction..........................................................................................................7

3PAR OS Versions................................................................................................................................7Audience...............................................................................................................................................7Supported Configurations.....................................................................................................................83PAR OS Upgrade Considerations.......................................................................................................83PAR Documentation...........................................................................................................................8

2 Configuring the 3PAR StoreServ Storage for FC................................................9Configuring the 3PAR StoreServ Storage Port Running 3PAR OS 3.2.x or 3.1.x ...............................9

Setting Up the Ports for an FC Connection...................................................................................10Creating a Host Definition for FC...................................................................................................12

Setting Up and Zoning the Fabric.......................................................................................................15FC Smart SAN...............................................................................................................................163PAR Coexistence.........................................................................................................................16Configuration Guidelines for FC Switch Vendors..........................................................................17Target Port Limits and Specifications for FC.................................................................................183PAR Priority Optimization for FC.................................................................................................193PAR Persistent Ports for FC........................................................................................................19

3PAR Persistent Ports Setup and Connectivity Guidelines for FC...........................................203PAR Persistent Checksum over FC..................................................................................................203PAR Express Writes..........................................................................................................................20

3 Configure the 3PAR StoreServ Storage for iSCSI.............................................21Setting Up the Ports for an iSCSI Connection....................................................................................213PAR iSCSI IPv6 Addressing and vLAN Tagging...............................................................................22Create a Host Definition for iSCSI......................................................................................................24

Creating a Host Definition for iSCSI..............................................................................................25Target Port Limits and Specifications for iSCSI..................................................................................263PAR Priority Optimization for iSCSI..................................................................................................263PAR Persistent Ports for iSCSI.........................................................................................................27

4 Configuring the 3PAR StoreServ Storage for FCoE..........................................28Setting Up the FCoE Switch, FCoE Initiator, and FCoE target ports..................................................28Create a Host Definition for FCoE......................................................................................................30

Creating a Host Definition for FCoE..............................................................................................32Target Port Limits and Specifications for FCoE..................................................................................333PAR Priority Optimization for FCoE..................................................................................................333PAR Persistent Ports for FCoE.........................................................................................................34

3PAR Persistent Ports Setup and Connectivity Guidelines for FCoE...........................................345 Configure the Host for an FC Connection.........................................................35

Installing the HBA and Drivers............................................................................................................35Install Virtual Machine Guest OS........................................................................................................36Multipath Failover Considerations and I/O Load Balancing................................................................38

Configure Round Robin Multipathing on ESX 4.x or later for FC..................................................40Configure ESX/ESXi Multipathing for Round Robin via SATP PSP..............................................42SATP Info Commands...................................................................................................................46

Default SATP Rules and Their Current Default PSP................................................................46SATP Custom Rules and Associated Defined Parameters......................................................47Showing Device Information....................................................................................................47Script Alternative for Path Policy Changes on Storage Devices without a Host Reboot.........48

Changing Path Policy on a Batch of LUNs Using CLI Commands.....................................48Performance Considerations for Multiple Host Configurations...........................................................49

Contents 3

ESX/ESXi Handling SCSI Queue Full and Busy Messages from the 3PAR StoreServ StorageArray..............................................................................................................................................49

VMware ESX Release ESX 4.x, ESXi 5.0 and 5.0 Updates, ESXi 5.5 and 5.5 Updates, andESXi 6.0...................................................................................................................................49

ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature Considerations.......................................51Storage I/O Control........................................................................................................................51VAAI (vStorage APIs for Array Integration)...................................................................................513PAR VAAI Plug-in 1.1.1 for ESX/ESXi 4.1...................................................................................523PAR VAAI Plug-in 2.2.0 for ESXi 5.x and ESXi 6.0.....................................................................52UNMAP (Space Reclaim) Storage Hardware Support for ESXi 5.x or ESXi 6.0...........................53Out-of-Space Condition for ESX 4.1, ESXi 5.x, or ESXi 6.0.........................................................54Additional New Primitives Beginning with ESXi 5.x.......................................................................56VAAI and New Feature Support Table...........................................................................................56VAAI Plug-in Verification................................................................................................................56

6 Configuring the Host as an FCoE Initiator Connecting to an FC target or anFCoE Target.........................................................................................................58

Configuring the FCoE Switch..............................................................................................................58Using system BIOS to configure FCoE...............................................................................................58Configure a 3PAR StoreServ Storage Port for an FCoE Host Connection.........................................61Configuring Initiator FCoE to FC Target..............................................................................................61Configuring Initiator FCoE to FCoE Target.........................................................................................63

7 Configure the Host for an iSCSI Connection.....................................................64Enterprise iSCSI.................................................................................................................................64Setting Up the Switch and iSCSI Initiator...........................................................................................65Installing Software iSCSI on VMware ESX.........................................................................................65Creating a VMkernel Port for Software iSCSI (ESX 4.0 Example).....................................................68Configuring a Service Console Connection for the iSCSI Storage on ESX 4.x..................................71Create a VMkernel Port for Software iSCSI........................................................................................74

Creating a VMkernel Port in the vSphere Client (ESXi 5.1 Example)...........................................74Configuring the VMware Software iSCSI Initiator...............................................................................76

Adding the VMware Software iSCSI Adapter (ESXi 5.1 Example)................................................76Setting Up and Configuring Challenge-Handshake Authentication Protocol......................................81Configuring CHAP on ESX/ESXi Host................................................................................................82Hardware iSCSI Support....................................................................................................................83

Independent Hardware iSCSI........................................................................................................83Adding an iSCSI Target............................................................................................................84

Installing Virtual Machine Guest Operating System...........................................................................87iSCSI Failover Considerations and Multipath Load Balancing...........................................................88Performance Considerations for Multiple Host Configurations...........................................................88ESX/ESXi Additional Feature Considerations....................................................................................88

8 Allocating Storage for Access by the ESX Host................................................89Create Storage on the 3PAR StoreServ Storage................................................................................89

Create Thinly Deduplicated Virtual Volumes.................................................................................89Create Virtual Volumes..................................................................................................................90

Exporting LUNs to the Host................................................................................................................92Create a VLUN for Export..............................................................................................................93

Exporting the LUNs..................................................................................................................94Discovering LUNs on VMware ESX Hosts.........................................................................................95Removing Volumes.............................................................................................................................96Host and Storage Usage—Eventlog and Host Log Messages...........................................................97

9 Boot the VMware ESX Host from the 3PAR StoreServ Storage.......................98Preparing the Host HBAs for a SAN Boot Deployment......................................................................98

4 Contents

10 Using VMware VVols (VMware Virtual Volumes) with 3PAR StoreServStorage.................................................................................................................99

Verify Licenses for VMware VVol Support........................................................................................101Verifying Installed Licenses.........................................................................................................101

VASA Provider..................................................................................................................................102VASA Provider Protocol Endpoints (PE) and 3PAR.........................................................................103VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.1 Setup...........................................105

Setting up VVols on the 3PAR StoreServ Storage with 3PAR OS 3.2.1.....................................105Understanding Certificates.....................................................................................................105Enabling the 3PAR VASA Provider........................................................................................106Assigning 3PAR Administrative User for VVols and Setting Up Storage Container...............107Allocate Common Provisioning Groups (CPG) for VVols on the array..................................108

Specifying VVol Capability Profiles...................................................................................108Setting Up Common Provisioning Groups........................................................................108The Special vvol_ CPG.....................................................................................................109Default 3PAR OS CPGs for VVols....................................................................................110

Registering the 3PAR VASA Provider with VMware vCenter.................................................111Additional Features with 3PAR OS 3.2.2..........................................................................................116VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.2 Set up..........................................116

Setting up VVols on the 3PAR StoreServ Storage with 3PAR OS 3.2.2.....................................116Managing Certificates............................................................................................................116

VASA Provider Certificate Management...........................................................................116Choosing the Certificate Management Mode ...................................................................117Server-Managed Certificate Management Mode..............................................................117Using 3PAR VASA Provider with a Public Key Infrastructure (PKI) .................................117Using 3PAR VASA Provider with a Self-Signed Certificate...............................................119Client-Managed Certificate Management Mode...............................................................119Using 3PAR VASA Provider with vSphere-Managed Certificate (VMCA) ........................120

Enabling the 3PAR VASA Provider........................................................................................120Assigning 3PAR Administrative User for VMware VVols........................................................121Creating and Viewing Storage Containers.............................................................................123Allocating Common Provisioning Groups (CPG) for VVols on the Array...............................124

Specifying VVol Capability Profiles...................................................................................124Setting Up Common Provisioning Groups........................................................................124The Special vvol_ CPG.....................................................................................................125Default 3PAR OS CPGs for VVols....................................................................................125

Enabling 3PAR Adaptive Flash Cache for use with VMware VVol (Optional)........................126Registering the VASA Provider (3PAR VASA Provider) with vCenter....................................126

Registering the Storage Provider (3PAR VASA Provider) with vCenter......................................12711 Configuring for 3PAR Peer Persistence.........................................................131

3PAR Peer Persistence....................................................................................................................13112 Host Persona Migration.................................................................................132

Host Persona Overview....................................................................................................................132Methods for Changing Host Persona................................................................................................132Preparing for Host Persona Migration..............................................................................................133Host Persona Pre-migration.............................................................................................................133

Finding 3PAR Custom SATP Rule...............................................................................................133Removing 3PAR Custom SATP Rules........................................................................................134

Migrating Using Online Method........................................................................................................134Migrating Using Offline Method........................................................................................................136

13 Support and other resources.........................................................................137Accessing Hewlett Packard Enterprise Support...............................................................................137Accessing updates............................................................................................................................137

Contents 5

Websites...........................................................................................................................................138Customer self repair.........................................................................................................................138Remote support................................................................................................................................138Documentation feedback..................................................................................................................138

Index...................................................................................................................139

6 Contents

1 IntroductionThis implementation guide provides information for establishing communication between anHPE 3PAR StoreServ Storage and a VMware ESX/ESXi host. General information is providedon the basic procedures for allocating storage on the 3PAR StoreServ Storage that is accessedby the host.

NOTE:• For predictable performance and results with the 3PAR StoreServ Storage, the information

in this guide must be used in concert with the documentation provided by Hewlett PackardEnterprise for the 3PAR StoreServ Storage and the documentation provided by the vendorfor their respective products.

• In addition to the OS patches mentioned in this guide, there might be additional patchesreferenced at the Storage Single Point of Connectivity Knowledge (SPOCK) website.

For information about supported hardware and software platforms, see the SPOCK website(SPOCK Home→Explore Storage Interoperability With SPOCK→Explore 3PAR StoreServStorage interoperability).

More informationhttp://www.hpe.com/storage/spock

3PAR OS VersionsThe following 3PAR OS versions are supported, listed in chronological release order:

3PAR OS 3.1.1, 3PAR OS 3.1.2, 3PAR OS 3.1.3, 3PAR OS 3.2.1, 3PAR OS 3.2.2

For example, where the guidance suggests 3PAR OS 3.1.x and later, it applies to 3PAR OS3.1.1, 3PAR OS 3.1.2, 3PAR OS 3.1.3, 3PAR OS 3.2.1, 3PAR OS 3.2.2, and any associatedMUs.

AudienceThis implementation guide is intended for system and storage administrators who monitor anddirect system configurations and resource allocation for the 3PAR StoreServ Storage.This guide provides basic information for establishing communication between the 3PAR StoreServStorage and the host and allocating the required storage for a given configuration. See theappropriate Hewlett Packard Enterprise documentation in conjunction with the host and HBA(Host Bus Adapter) documentation for specific details and procedures.

3PAR OS Versions 7

http://www.hpe.com/storage/spock

Supported ConfigurationsThe following types of host connections are supported between the 3PAR StoreServ Storageand hosts with an ESX OS:

• FC (Fibre Channel)

• iSCSI

As Software iSCSI initiator◦◦ As Hardware iSCSI initiator

• FCoE (Fibre Channel over Ethernet)

◦ FCoE initiator connecting to an FC target or an FCoE target

FC connections are supported between the 3PAR OS and the host in both a fabric-attached anddirect-connect topology.

3PAR OS Upgrade ConsiderationsThis implementation guide refers to new installations. For information about planning an online3PAR OS upgrade, see the HPE 3PAR Operating System Upgrade Pre-Planning Guide at theHewlett Packard Enterprise Storage Information Library.For complete details about supported host configurations and interoperability, see the SPOCKwebsite (SPOCK Home→Explore Storage Interoperability With SPOCK→Explore 3PARStoreServ Storage interoperability.

More informationhttp://www.hpe.com/info/storage/docshttp://www.hpe.com/storage/spock

3PAR Documentation

See:Type of Documentation:

The Hewlett Packard Enterprise Storage Information Library:Current version of this implementation guideand additional 3PAR storage documentation: http://www.hpe.com/info/storage/docs

The Single Point of Connectivity Knowledge (SPOCK) for HewlettPackard Enterprise Storage Products website:

Supported hardware and software platforms:


The Hewlett Packard Enterprise Customer Self Repair ServicesMedia Library:

Customer Self Repair procedures (media):

http://www.hpe.com/support/csrUnder Product category, select Storage. Under Product family,select 3PAR StoreServ Storage for HPE 3PAR StoreServ 7000,8000, 10000, and 20000 Storage systems.

Hewlett Packard Enterprise Support Center:All Hewlett Packard Enterprise products:http://www.hpe.com/support/hpesc

8 Introduction

http://www.hpe.com/info/storage/docs




http://www.hpe.com/support/csr

http://www.hpe.com/support/hpesc

2 Configuring the 3PAR StoreServ Storage for FCThis topic explains how to establish an FC connection between the 3PAR StoreServ Storageand a host and covers 3PAR OS versions 3.2.x and 3.1.x. For information on setting up thephysical connection for a particular 3PAR StoreServ Storage, see the appropriate installationmanual.

Configuring the 3PAR StoreServ Storage Port Running 3PAR OS 3.2.xor 3.1.x

Before connecting the 3PAR StoreServ Storage port to the host, configure the ports using theprocedures in the following topics.When deploying Virtual Connect Direct-Attach Fibre Channel for 3PAR StoreServ Storagesystems, where the 3PAR StoreServ Storage ports are cabled directly to the uplink ports oneither the Virtual Connect FlexFabric 10 Gb/24-port Module for c-Class BladeSystem or VirtualConnect FlexFabric 20/40 F8 Module for c-Class BladeSystem, follow the procedure for configuringthe 3PAR StoreServ Storage ports for a fabric connection.For more information about Virtual Connect, Virtual Connect interconnect modules, and theVirtual Connect Direct-Attach Fibre Channel feature, see Virtual Connect documentation. Toobtain this documentation, search the Hewlett Packard Enterprise Support Center website.When using Brocade FC HBAs, download Brocade HBA drivers, firmware, and the BCU utilityfrom the QLogic website. Specific Brocade drivers are named with a BR prefix, such as BR-xxxHBA model.By default, the QLogic, Emulex, and Brocade drivers for the ESX server support failover. Forfailover support using the QLogic, Emulex, or Brocade driver, VVs should be simultaneouslyexported down multiple paths to the host. To do this, create a host definition on the 3PARStoreServ Storage that includes the WWNs of multiple HBA ports on the host, and then exportthe VLUNs to that host definition. If each server within a cluster has its own host definition, theVLUNs must be exported to multiple host definitions.See the HPE SAN Design Reference Guide at the SPOCK website (SPOCK Home→DesignGuides→SAN Design Guide.

More informationhttp://www.hpe.com/support/hpeschttp://www.qlogic.comhttp://www.hpe.com/storage/spock

Configuring the 3PAR StoreServ Storage Port Running 3PAR OS 3.2.x or 3.1.x 9


http://www.qlogic.com


Setting Up the Ports for an FC ConnectionBefore connecting the 3PAR StoreServ Storage to a host, specify either an FC direct connectionor an FC fabric connection type and mode for each port:1. Determine if a port is configured in host mode:

cli % showport -par

N:S:P Connmode ConnType CfgRate MaxRate Class2 UniqNodeWwn VCN IntCoal TMWO Smart_SAN2:1:1 disk loop auto 16Gbps disabled disabled disabled enabled disabled unsupported

A host port is essentially a target mode port where the initiator or host can log in to the 3PARStoreServ Storage.

2. If the port was not configured, take the port offline before configuring it for the host:

# controlport offline <node:slot:port>

CAUTION: Before taking a port offline in preparation for a direct or fabric connection,verify that the port has not been previously defined and that it is not already connected to ahost as this would interrupt the existing host connection.If a 3PAR StoreServ Storage port is already configured for a direct or fabric connection,ignore this step, because it is not necessary to take the port offline.

3. Configure the port for the host by using the -ct parameter:

# controlport config host -ct [loop | point] <node:slot:port>

FC Direct Connect:A different connection type is used for direct connection to 8 Gb FC array target port typesversus 16 Gb FC array target port types.

NOTE: Beginning with 3PAR OS 3.2.2 MU2, direct connect for the 16 Gb FC 3PARStoreServ Storage target interface is supported. Support for direct connect to the 16 Gb FC3PAR StoreServ Storage target interface is limited to specific host initiators. For moreinformation, see the SPOCK website (SPOCK Home→Explore Storage InteroperabilityWith SPOCK→Explore 3PAR StoreServ Storage interoperability).

To identify the type of array target port adapter, use the showport -par command andidentify the MaxRate listed for the port planned for direct connect.For example, array target port 0:1:1 is identified as 8Gbps and 2:1:1 as 16Gbps:

cli % showport -par

N:S:P Connmode ConnType CfgRate MaxRate Class2 UniqNodeWwn VCN IntCoal TMWO Smart_SAN0:1:1 host loop auto 8Gbps disabled disabled disabled disabled enabled n/a1:1:1 host loop auto 8Gbps disabled disabled disabled disabled enabled n/a2:1:1 host point auto 16Gbps disabled disabled disabled disabled disabled unsupported3:1:1 host point auto 16Gbps disabled disabled disabled disabled disabled unsupported

• For a Direct Connection to an 8 Gb FC 3PAR StoreServ Storage targetinterface—Use the -ct loop parameter to specify a direct connection.

• For a Direct Connection to a 16 Gb FC 3PAR StoreServ Storage targetinterface—Use the -ct point parameter to specify a direct connection.

10 Configuring the 3PAR StoreServ Storage for FC

FC Fabric Connect:For all FC fabric connections, use the -ct point parameter to specify a fabric connection.

4. Use the controlport rst <node:slot:port> command to reset and register the newport definitions.The following example shows how to set up a fabric connected port.

cli % controlport offline 2:1:1cli % controlport config host -ct point 2:1:1cli % controlport rst 2:1:1

After all the ports are configured, verify that they are configured for a host in a directconnection by using the showport -par command on the 3PAR StoreServ Storage.

cli % showport -par

N:S:P Connmode ConnType CfgRate MaxRate Class2 UniqNodeWwn VCN IntCoal TMWO Smart_SAN2:1:1 host point auto 16Gbps disabled disabled disabled disabled disabled unsupported




Creating a Host Definition for FC

Create a Host Definition for FC OverviewBefore connecting the host to the 3PAR StoreServ Storage, a host definition needs to be createdthat specifies a valid host persona for each 3PAR StoreServ Storage port that is to be connectedto a host HBA port through a fabric or direct connection.• With 3PAR OS 3.1.1 or earlier, ESX/ESXi uses the host Generic-Legacy Persona 6.

For any version of VMware ESX deployed with 3PAROS versions after 3PAROS 3.1.3,host Generic-Legacy Persona 6 will not be supported.With 3PAR OS 3.1.2 or later, Hewlett Packard Enterprise recommends migrating the ESXconfigurations to host VMware Persona 11.For details about migrating the host from host Generic-Legacy Persona 6 to host VMwarePersona 11, see “Host Persona Migration” (page 132).

NOTE: When changing an existing host persona from Generic-Legacy Persona 6 toVMware Persona 11, a host reboot is required tor the change to take effect. This is an offlineprocess. The host persona change should coincide with changing the SATP rules on thehost as well.

• Beginning with 3PAR OS 3.1.2, a second host VMware Persona 11, which enablesasymmetric logical unit access (ALUA) is available.

◦ Host VMware Persona 11 is recommended for new ESX/ESXi installations and isrequired for ESX/ESXi hosts configured as part of 3PAR Peer Persistence and VMwarevSphere Metro Storage Cluster (vMSC) configurations.

◦ To configure 3PAR Peer Persistence with Automatic Transparent Failover and ManualTransparent Switchover, see the HPE 3PAR Remote Copy Software User Guide at theHewlett Packard Enterprise Storage Information Library website.

◦ To implement vSphere Metro Storage Cluster (vMSC) using 3PAR Peer Persistence,see KB 2055904 at the VMware Knowledge Base website.

• ESX/ESXi hosts performing 3PAR single volume Peer Motion is required to use 3PAR OS3.1.3 or later and host VMware Persona 11.

• For ESX/ESXi hosts with 3PAR Remote Copy, see the 3PAR Remote Copy Software UserGuide for the appropriate host persona to use in specific Remote Copy configurations at theHewlett Packard Enterprise Storage Information Library website.


For both host Generic-Legacy Persona 6 and VMware Persona 11, see the appropriate topicsin this guide for FC setup considerations.

NOTE:Beginning with 3PAR OS 3.2.1, an in-band device known as a PE LUN (Protocol EndpointLUN) is exposed to all hosts connected using VMware Persona 11. The PE LUN is presentedas part of the support for VMware vSphere 6.0 VVols. Depending on the host HBA/CNA driverin use, the PE LUN might appear as a LUN number 256 in the host device list. In addition, awarning message related to the PE LUN might appear in the ESXi 6.0 vmkernel logs when usingdrivers that do not support VVols. However, the error does not interfere with normal, non-VVoloperation.

For complete details about using the controlport, createhost, and showhost commands,see theHPE 3PARCommand Line Interface Reference or theHPE 3PARStoreServ ManagementConsole User’s Guide available at the Hewlett Packard Enterprise Storage Information Library.

More informationhttp://www.hpe.com/info/storage/docshttp://kb.vmware.com



http://kb.vmware.com

Creating a Host Definition for FC1. Display available host personas:

cli % showhost -listpersona

2. Create host definitions by using the createhost command with the -persona option tospecify the host persona and the host name.Beginning with 3PAR OS 3.2.2, the createhost command has been enhanced with the-port option, which automatically creates zones on the switch or fabric if the target port isa 16 Gb FC target and the switch supports enhanced zoning. See “FC Smart SAN” (page16).With 3PAR OS 3.1.1 or earlier:

cli % createhost -persona 6 ESXserver1 10000000C9724AB2 10000000C97244FE

With 3PAR OS 3.1.2 or later:

cli % createhost

Id Name Persona -WWN/iSCSI_Name- Port0 ESXserver1 Generic-legacy 10000000C9724AB2 ---

10000000C97244FE ---

3. Verify that the host has been created by using the showhost command.With 3PAR OS 3.1.1 or earlier, using host Generic-Legacy Persona 6:

cli % showhost

Id Name Persona -WWN/iSCSI_Name- Port0 ESXserver1 Generic-legacy 10000000C9724AB2 ---

10000000C97244FE ---

With 3PAR OS 3.1.2 or later, using host VMware Persona 11:

cli % showhost

Id Name Persona -WWN/iSCSI_Name- Port0 ESXserver2 VMware 100000051EC33E00 ---

100000051EC33E01 ---

Use showhost -persona to show the persona name and Id relationship.

cli % showhost -persona

Id Name Persona_Id Persona_Name Persona_Caps0 ESXserver1 6 Generic-legacy --1 Esxserver2 11 VMware SubLun, ALUA


Setting Up and Zoning the FabricNOTE: This topic does not apply when deploying Virtual Connect Direct-Attach Fibre Channelfor 3PAR StoreServ Storage systems, where the 3PAR StoreServ Storage ports are cableddirectly to the uplink ports on either the Virtual Connect FlexFabric 10 Gb/24-port Module forc-Class BladeSystem or Virtual Connect FlexFabric 20/40 F8 Module for c-Class BladeSystem.Zoning is automatically configured based on the Virtual Connect SAN Fabric and server profiledefinitions.For more information about Virtual Connect, Virtual Connect interconnect modules, and theVirtual Connect Direct-Attach Fibre Channel feature, see the Hewlett Packard Enterprise SupportCenter website.See also theHPESANDesign Reference Guide at the SPOCK website (SPOCKHome→DesignGuides→SAN Design Guide).

Fabric zoning controls which FC end-devices have access to each other on the fabric. Zoningalso isolates the host and 3PAR StoreServ Storage ports from Registered State ChangeNotifications (RSCNs) that are irrelevant to these ports.Set up fabric zoning by associating the device World Wide Names (WWNs) or the switch portswith specified zones in the fabric. Use either the WWN method or the port zoning method withthe 3PAR StoreServ Storage. The WWN zoning method is recommended because the zonesurvives the changes of switch ports when cables are moved around on a fabric.Required:Employ fabric zoning, by using the methods provided by the switch vendor, to create relationshipsbetween host HBA/CNA ports and 3PAR StoreServ Storage ports before connecting the hostHBA/CNA ports or 3PAR StoreServ Storage ports to the fabrics.FC switch vendors support the zoning of the fabric end-devices in different zoning configurations.There are advantages and disadvantages with each zoning configuration, so determine what isneeded before choosing a zoning configuration.The 3PAR StoreServ Storage arrays support the following zoning configurations:

• One initiator to one target per zone

• One initiator to multiple targets per zone (zoning by HBA). This zoning configuration isrecommended for the 3PAR StoreServ Storage. Zoning by HBA is required for coexistencewith other Hewlett Packard Enterprise storage systems.

NOTE:

◦ For high availability and clustered environments that require multiple initiators to accessthe same set of target ports, Hewlett Packard Enterprise recommends creating separatezones for each initiator with the same set of target ports.

◦ The storage targets in the zone can be from the same 3PAR StoreServ Storage, multiple3PAR StoreServ Storages, or a mixture of 3PAR and other Hewlett PackardEnterprise storage systems.

Setting Up and Zoning the Fabric 15

For more information about using one initiator to multiple targets per zone, see the HPE SANDesign ReferenceGuide at the SPOCK website (SPOCKHome→DesignGuides→SANDesignGuide).When using an unsupported zoning configuration and an issue occurs, Hewlett PackardEnterprise might require implementing one of the supported zoning configurations as part of thecorrective action.Verify the switch and zone configurations by using the 3PAR CLI showhost command to verifythat each initiator is zoned with the correct targets after completing the following tasks:

• Complete configuration of the storage port to the host and connect to the switch.

• Create a zone configuration on the switch following the HPE SAN Design Reference Guideand enable the zone set configuration.

• Use the showhost command to verify that the host is seen on the storage node.

More informationhttp://www.hpe.com/support/hpeschttp://www.hpe.com/storage/spock

FC Smart SANBeginning with 3PAR OS 3.2.2, the following 3PAR StoreServ Storage systems support SmartSAN on 16 Gb FC targets:

• 3PAR StoreServ 20000 Storage



• 3PAR StoreServ 7000 StorageSmart SAN for 3PAR through its TDPZ (target-driven peer zoning) feature enables customersto automate peer zoning, which results in the creation of fewer zones and enables configurationof zones in minutes. Through automation, it reduces the probability of errors and potentialdowntime. Without Smart SAN, an administrator needs to preconfigure zones on the FC switch,before configuring hosts and VLUNs on the 3PAR StoreServ Storage. With Smart SAN, theadministrator can configure and control zoning directly from the 3PAR CLI.For information about supported FC switches and their firmware revisions with Smart SAN, seethe SPOCK website.For more information about Smart SAN for 3PAR, including configuration, see the HPE 3PARSmart SAN 1.0 User Guide at the Hewlett Packard Enterprise Storage Information Library website.

More informationhttp://www.hpe.com/storage/spockhttp://www.hpe.com/info/storage/docs

3PAR CoexistenceThe 3PAR StoreServ Storage array can coexist with other Hewlett Packard Enterprise storagearray families.For supported Hewlett Packard Enterprise storage array combinations and rules, see theHPESANDesign ReferenceGuide at the SPOCK website (SPOCKHome→DesignGuides→SANDesignGuide).








Configuration Guidelines for FC Switch VendorsUse the following FC switch vendor guidelines before configuring ports on fabrics to which the3PAR StoreServ Storage connects.

• Brocade switch ports that connect to a host HBA port or to a 3PAR StoreServ Storage portshould be set to their default mode. On Brocade 3xxx switches running Brocade firmware3.0.2 or later, verify that each switch port is in the correct mode by using the Brocade telnetinterface and the portcfgshow command, as follows:

brocade2_1:admin> portcfgshow

Ports 0 1 2 3 4 5 6 7-----------------+--+--+--+--+----+--+--+--Speed AN AN AN AN AN AN AN ANTrunk Port ON ON ON ON ON ON ON ONLocked L_Port .. .. .. .. .. .. .. ..Locked G_Port .. .. .. .. .. .. .. ..Disabled E_Port .. .. .. .. .. .. .. ..where AN:AutoNegotiate, ..:OFF, ??:INVALID.

The following fill-word modes are supported on a Brocade 8 Gb switch running FOS firmware6.3.1a and later:

admin>portcfgfillword

Usage: portCfgFillWord PortNumber Mode [Passive]Mode: 0/-idle-idle - IDLE in Link Init, IDLE as fill word (default)

1/-arbff-arbff - ARBFF in Link Init, ARBFF as fill word2/-idle-arbff - IDLE in Link Init, ARBFF as fill word (SW)3/-aa-then-ia - If ARBFF/ARBFF failed, then do IDLE/ARBFF

Hewlett Packard Enterprise recommends setting the fill word to mode 3 (aa-then-ia),which is the preferred mode, by using the portcfgfillword command. If the fill word isnot correctly set, er_bad_os counters (invalid ordered set) will increase when using theportstatsshow command while connected to 8 Gb HBA ports, as they need theARBFF-ARBFF fill word. Mode 3 will also work correctly for lower-speed HBAs, such as 4Gb/2 Gb HBAs. For more information, see the Fabric OS Command Reference Manual andthe FOS release notes, at the Brocade website.

NOTE: In addition, some Hewlett Packard Enterprise switches, such as the HPE SN8000B8-slot SAN backbone director switch, the HPE SN8000B 4-slot SAN director switch, theHPE SN6000B 16 Gb FC switch, or the HPE SN3000B 16 Gb FC switch automatically selectthe proper fill-word mode 3 as the default setting.

• McDATA switch or director ports should be in their default modes asG orGX-port (dependingon the switch model), with their speed setting permitting them to autonegotiate.

• Cisco switch ports that connect to 3PAR StoreServ Storage ports or host HBA ports shouldbe set to AdminMode = FX and AdminSpeed = auto port, with the speed set to autonegotiate.

• QLogic switch ports should be set to port type GL-port and port speed auto-detect. QLogicswitch ports that connect to the 3PAR StoreServ Storage should be set to I/O Stream Guarddisable or auto, but never enable.

More informationhttp://www.brocade.com/en.html


http://www.brocade.com/en.html

Target Port Limits and Specifications for FCTo avoid overwhelming a target port and to ensure continuous I/O operations, observe thefollowing limitations on a target port:

• Follow the instructions for setting the maximum number of initiator connections supportedper array port, per array node pair, and per array as shown in the HPE 3PAR Support Matrixdocumentation at the SPOCK website (SPOCK Home→Other Hardware→3PAR).

• Maximum I/O queue depth per port on each 3PAR StoreServ Storage HBA model, as follows:

Max. QueueDepthPortsSpeedBusArrayProtocolHBA

95924 GbpsPCI-XF200, F400,T400, T800

FCEmulexLP11002

163844 GbpsPCI-XF200, F400,T400, T800

FC3PARFC044X

327628 GbpsPCIe3PARStoreServ

FCEmulexLPe12002

7000


FCEmulexLPe12004

7000, 10000


FCEmulexLPe16002

7000, 8000,10000


FCEmulexLPe16004

8000, 20000

• The I/O queues are shared among the connected host HBA ports on a first-come, first-servedbasis.

• When all queues are in use and a host HBA port tries to initiate I/O, it receives a target queuefull response from the 3PAR StoreServ Storage port. This condition can result in erratic I/Operformance on each host. If this condition occurs, each host should be throttled so that itcannot overrun the 3PAR StoreServ Storage port's queues when all hosts are deliveringtheir maximum number of I/O requests.

NOTE:

◦ When host ports can access multiple targets on fabric zones, the target number assignedby the host driver for each discovered target can change when the host is booted andsome targets are not present in the zone. This situation might change the device nodeaccess point for devices during a host restart. This issue can occur with anyfabric-connected storage, and is not specific to the 3PAR StoreServ Storage.

◦ The maximum number of I/O paths supported is 16.




3PAR Priority Optimization for FCBeginning with 3PAR OS 3.1.2 MU2, the 3PAR Priority Optimization feature was introduced.This feature is a more efficient and dynamic solution for managing server workloads and can beutilized as an alternative to setting host I/O throttles. When using this feature, a storageadministrator is able to share storage resources more effectively by enforcing quality of servicelimits on the array.SCSI queue full handling must be enabled on VMware hosts before enabling 3PAR PriorityOptimization on the array. For details, see “ESX/ESXi Handling SCSI Queue Full and BusyMessages from the 3PAR StoreServ Storage Array” (page 49). Other per target or per adapterthrottle settings might need to be adjusted in rare cases.For complete details of how to use 3PAR Priority Optimization (Quality of Service) on 3PARStoreServ Storage systems, see the HPE 3PAR Priority Optimization technical whitepaper:http://www.hpe.com/info/3PAR-Priority-Optimization

3PAR Persistent Ports for FCThe 3PAR Persistent Ports (or virtual ports) feature minimizes I/O disruption during a 3PARStoreServ Storage online upgrade or node-down event. Port shutdown or reset events do nottrigger this feature.Each FC target storage array port has a partner array port automatically assigned by the system.Partner ports are assigned across array node pairs.3PAR Persistent Ports allows a 3PAR StoreServ Storage FC port to assume the identity of afailed port (WWN port) while retaining its own identity. Where a given physical port assumes theidentity of its partner port, the assumed port is designated as a persistent port. Array port failoverand failback with 3PAR Persistent Ports is transparent to most host-based multipathing software,which can keep all of its I/O paths active.

NOTE: Use of 3PAR Persistent Ports technology does not negate the need for properly installed,configured, and maintained host multipathing software.

For a more complete description of the 3PAR Persistent Ports feature, its operation, and acomplete list of required setup and connectivity guidelines, see the following documents:

• Technical whitepaper HPE 3PAR StoreServ Persistent Ports (Hewlett PackardEnterprise document #F4AA4-4545ENW) at the Hewlett Packard Enterprise Support Centerwebsite.

• HPE 3PAR Command Line Interface Administrator’s Manual in the "Using Persistent Portsfor Nondisruptive Online Software Upgrades" topic at the Hewlett Packard Enterprise StorageInformation Library website.

More informationhttp://www.hpe.com/support/hpeschttp://www.hpe.com/info/storage/docs


http://www.hpe.com/info/3PAR-Priority-Optimization



3PAR Persistent Ports Setup and Connectivity Guidelines for FCBeginning with 3PAR OS 3.1.2, the 3PAR Persistent Ports feature is supported for FC targetports.Beginningwith 3PAROS 3.1.3, the Persistent Port feature has additional functionality to minimizeI/O disruption during an array port loss_sync event triggered by a loss of array port connectivityto the fabric.Follow the specific cabling setup and connectivity guidelines so that 3PAR Persistent Portsfunction properly:

• 3PAR StoreServ Storage FC partner ports must be connected to the same FC fabric, andpreferably to different FC switches on the fabric.

• The FC fabric must support NPIV, and NPIV must be enabled.

• Configure the host-facing HBAs for point-to-point fabric connection (there is no support fordirect-connect "loops").

For information regarding the Persistent Ports feature for an FCoE initiator to FC targetconfiguration (FCoE to FC switched), see “Configuring the 3PAR StoreServ Storage for FC”(page 9).

3PAR Persistent Checksum over FCThe 3PAR Persistent Checksum feature, available on 3PAR StoreServ 20000 and 8000 Storagesystems (arrays), provides end-to-end data integrity protection from the host initiator HBA throughthe data network to the backend drives on the StoreServ storage system. This feature is basedon the DIF protection model defined by the SCSI T10 committee. It provides data protection fromsilent data corruption of any media and transmission errors caused by any component in the I/Ostack across the data network. Detection and recovery features have been built into the 3PAROS to correct issues discovered through this feature.Minimum host HBA driver versions for support of 3PAR Persistent Checksum over FC arerequired.With ESXi 5.5 and later, supporting drivers are available. With these drivers, the 3PARPersistent Checksum on 3PAR StoreServ Storage arrays is enabled by default, and there is noneed to configure anything on the array or the host HBA driver.For supported HBAs, driver version, and supported host OSs that support 3PAR PersistentChecksum (DIF capable), see SPOCK website.For additional information on the 3PAR Persistent Checksum, see the HPE 3PAR StoreServArchitecture technical whitepaper at the Hewlett Packard Enterprise Storage Information Librarywebsite.

More informationhttp://www.hpe.com/storage/spockhttp://www.hpe.com/info/storage/docs

3PAR Express WritesBeginning with 3PAROS 3.2.1, the Express Writes feature was introduced and is only availableon the 3PAR StoreServ 7000 and 10000 Storage systems with 8 Gb targets and not supportedon 3PAR StoreServ 20000, 10000, 8000, or 7000 Storage systems with 16 Gb targets. Thisfeature optimizes the performance for small block random writes and is enabled by default with8 Gb Targets in Host HBA mode with bit Express Writes.




3 Configure the 3PAR StoreServ Storage for iSCSIThis topic explains how to establish an iSCSI connection between the 3PAR StoreServ Storageand the VMware ESX host. With specific CNA cards, a Software iSCSI or Hardware iSCSI initiatorcan be used.

NOTE: Hewlett Packard Enterprise recommends using default values to configure the 3PARStoreServ Storage host, unless otherwise specified in the following procedures.

Setting Up the Ports for an iSCSI ConnectionTo establish an iSCSI connection between a 3PAR StoreServ Storage system and the ESX host,set up each 3PAR StoreServ Storage iSCSI target port that will be connected to an iSCSI initiatoras described in the following procedure:1. Set up a one-time configuration for the iSCSI ports on the 3PAR StoreServ Storage by using

the 3PAR CLI controlport config iscsi <n:s:p> command.First, use the showport and showport -i commands to check the current CNAconfiguration. For example:

# showport

N:S:P Mode State -Node_WWN-- -Port_WWN/HW_Addr- Type Protocol Label Partner FailoverState0:1:1 target offline - 2C27D754521E iscsi iSCSI - - -1:1:1 target offline - 2C27D754521A iscsi iSCSI - - -

# showport -i

N:S:P Brand Model Rev Firmware Serial HWType0:1:1 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3SK CNA1:1:1 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3SK CNA

If State=config_wait or Firmware=0.0.0.0, use the 3PAR CLI controlportconfig iscsi <n:s:p> command to configure, and then use the showport andshowport -i commands to verify the configuration setting.For example:

# controlport config iscsi 0:1:1# controlport config iscsi 1:1:1# showport

N:S:P Mode State -Node_WWN-- -Port_WWN/HW_Addr- Type Protocol Label Partner FailoverState0:1:1 target ready - 2C27D754521E iscsi iSCSI - - -1:1:1 target ready - 2C27D754521A iscsi iSCSI - - -

# showport -i

...N:S:P Brand Model Rev Firmware Serial HWType...0:1:1 QLOGIC QLE8242 58 4.8.76.48015 PCGLT0ARC1K3SK CNA1:1:1 QLOGIC QLE8242 58 4.8.76.48015 PCGLT0ARC1K3SK CNA

2. Use the showport -iscsi command to check if configuration is required for the iSCSItarget ports.

# showport -iscsi

N:S:P State IPAddr Netmask Gateway TPGT MTU Rate DHCP iSNS_Prim iSNS_Sec iSNS_Port0:1:1 offline 0.0.0.0 0.0.0.0 0.0.0.0 11 1500 n/a 0 0.0.0.0 0.0.0.0.0 32051:1:1 offline 0.0.0.0 0.0.0.0 0.0.0.0 11 1500 n/a 0 0.0.0.0 0.0.0.0.0 3205

Setting Up the Ports for an iSCSI Connection 21

3. If configuration is needed, use the 3PAR CLI controliscsiport addr <ipaddr><netmask> [-f] <node:slot:port> command:

# controliscsiport addr 10.1.1.100 255.255.255.0 -f 0:1:1# controliscsiport addr 10.1.1.102 255.255.255.0 -f 1:1:1

NOTE:• For information on host port per target port and max total host ports per array, refer to

the HPE 3PAR Support Matrix documentation at the SPOCK website.• When the host initiator port and the 3PAR StoreServ Storage iSCSI target port are in

different IP subnets, configure the gateway address for the 3PAR StoreServ StorageiSCSI port to avoid unexpected behavior as follows:controliscsiport gw <gw_address> [-f] <node:slot:port>

4. Use the 3PAR CLI showport -iscsi command to verify the settings. If not completedpreviously, configure the ESX host iSCSI initiator according to “Configure the Host for aniSCSI Connection” (page 64).

5. From the ESX host, use the vmkping command to verify communication with the 3PARStoreServ Storage iSCSI target ports as shown in the following example:

# vmkping 10.1.1.100

6. From the 3PAR StoreServ Storage array, use the controliscsiport ping <ipadd><node:slot:port> command to verify communication with the ESX host initiator portsas shown in the following example:

# controliscsiport ping 10.1.1.10 0:1:1


3PAR iSCSI IPv6 Addressing and vLAN TaggingTo configure the 3PAR StoreServ Storage ports for iSCSI personality, follow these steps:1. Issue the following command:

cli % controlport config iscsi -f N:S:P

2. Check the current iSCSI port settings by issuing the showport -iscsi command. Thefollowing example shows ports offline and unconfigured (no IP address or vLAN tag) afterthe ports have been set up with the iSCSI personality in the previous step.

cli % showport -iscsi

N:S:P State IPAddr Netmask/PrefixLen Gateway TPGT MTU Rate iSNS_Addr iSNS_Port STGT VLAN0:6:1 offline :: 0 :: 61 1500 10Gbps :: 3205 61 -0:6:2 offline :: 0 :: 62 1500 10Gbps :: 3205 62 -1:6:1 offline :: 0 :: 161 1500 10Gbps :: 3205 161 -1:6:2 offline :: 0 :: 162 1500 10Gbps :: 3205 162 -----------------------------------------------------------------------------------------------------4

22 Configure the 3PAR StoreServ Storage for iSCSI


3. Issue the controliscsiport addr <netmask|prefix_len> [-f]<node:slot:port> command to set the IP addresses, prefix length, and vlan tag of theiSCSI target ports. For example:

cli % controliscsiport addr fd15:0816:c0::021 64 -vlan 816 -f 0:6:1

4. To display the IPv6 address and vlan tag of port 0:6:1, use the iscsivlan option in theshowport command. For example:

cli % showport -iscsivlans 0:6:1

N:S:P VLAN IPAddr Netmask/PrefixLen Gateway MTU TPGT STGT iSNS_Addr iSNS_Port0:6:1 816 fd15:816:c0::21 64 :: 1500 61 61 :: 3205

To see that port 0:6:1 is ready, issue the showport -iscsi command:

cli % showport -iscsi 0:6:1

N:S:P State IPAddr Netmask/PrefixLen Gateway TPGT MTU Rate iSNS_Addr iSNS_Port STGT VLAN0:6:1 ready :: 0 :: 61 1500 10Gbps :: 3205 61 Y--------------------------------------------------------------------------------------------------1

If configured with vLAN, use showport -iscsivlans instead of showport -iscsi todisplay IP Address:

cli % showport -iscsivlans

N:S:P VLAN IPAddr Netmask/PrefixLen Gateway MTU TPGT STGT iSNS_Addr iSNS_Port0:6:1 - 10.101.1.61 255.255.255.0 10.101.1.1 1500 61 61 0.0.0.0 32050:6:2 - :: 0 :: 1500 62 62 :: 32050:6:2 62 fd15:816:c0:21 64 :: 1500 62 62 :: 32051:6:1 - 10.101.2.161 255.255.255.0 10.101.2.1 1500 161 161 0.0.0.0 32051:6:2 - :: 0 :: 1500 162 162 :: 32051:6:2 162 fd15:1816:c0:21 64 :: 1500 162 162 :: 3205----------------------------------------------------------------------------------------------6

NOTE:To configure IPv4 addressing and vLAN tagging on the 3PAR StoreServ Storage systems, followthe steps in “Setting Up the Ports for an iSCSI Connection” (page 21), which describe how toconfigure IPv4 for the 3PAR StoreServ Storage systems, with one exception: you must use the-vlan option when configuring an IPv4 address with the vLAN number that you want to assignto the IPv4 address.

3PAR iSCSI IPv6 Addressing and vLAN Tagging 23

Create a Host Definition for iSCSICreate a Host Definition for iSCSI OverviewCreate a host definition that ties all of the connections from a single host to a host name. Beforecreating a host definition, the 3PAR StoreServ Storage iSCSI target ports must be set up andan iSCSI connection established. The iSCSI connection is established by following the procedurein “Setting Up the Ports for an iSCSI Connection” (page 21) and the procedure in “Configure theHost for an iSCSI Connection” (page 64) through “Configuring the VMware Software iSCSIInitiator” (page 76) (ESX host setup).

• With 3PAR OS 3.1.1 or earlier, ESX/ESXi uses the host Generic-Legacy Persona 6.With 3PAR OS versions after 3PAR OS 3.1.3, host Generic-Legacy Persona 6 will not besupported for any version of VMware ESX deployed.With 3PAR OS 3.1.2 or later, Hewlett Packard Enterprise recommends migrating the ESXconfigurations to host VMware Persona 11. For details about migrating the ESX host fromhost Generic-Legacy Persona 6 to host VMware Persona 11, see “Host Persona Migration”(page 132).

NOTE: When changing an existing host persona from 6 to 11, a host reboot is requiredtor the change to take effect. This is an offline process. The host persona change shouldcoincide with changing the SATP rules on the host as well.

• Beginning with 3PAR OS 3.1.2, a second host VMware Persona 11 enables asymmetriclogical unit access (ALUA).


◦ To configure 3PAR Peer Persistence with Automatic Transparent Failover and ManualTransparent Switchover, see the HPE 3PAR Remote Copy Software User Guide in theHewlett Packard Enterprise Storage Information Library website.


• ESX/ESXi hosts performing 3PAR single-volume Peer Motion is required to use 3PAR OS3.1.3 or later and host VMware Persona 11.

• For ESX/ESXi hosts with 3PAR Remote Copy, see the HPE 3PAR Remote Copy SoftwareUser Guide for the appropriate host persona to use in specific Remote Copy configurationsat the Hewlett Packard Enterprise Storage Information Library website.

NOTE:Beginning with the 3PAR OS 3.2.1, an in-band device known as a PE LUN (Protocol EndpointLUN) is exposed to all hosts connected using VMware Persona 11. The PE LUN is presentedas part of the support for VMware vSphere 6.0 VVols. Depending on the host HBA/CNA driverin use, the PE LUN might appear as a LUN number 256 in the host device list. In addition, awarning message related to the PE LUN might appear in the ESXi 6.0 vmkernel logs when usingdrivers that do not support VVols. However, the error does not interfere with normal, non-VVoloperation.





Creating a Host Definition for iSCSIThe following example for creating a host definition uses a VMware iSCSI initiator(iqn.1998-01.com.vmware:dl360g8-02-42b20fff) on an ESX host connecting througha VLAN to a pair of 3PAR StoreServ Storage iSCSI ports.1. Verify that the host iSCSI initiators are connected to the 3PAR StoreServ Storage iSCSI

target ports by using the 3PAR CLI showhost command.

# showhost

Id Name Persona ----------------WWN/iSCSI_Name---------------- Port-- -- iqn.1998-01.com.vmware:dl360g8-02-42b20fff 0:1:2-- -- iqn.1998-01.com.vmware:dl360g8-02-42b20fff 1:1:2

2. Create the appropriate host definition entry by using the 3PAR CLI createhost -iscsi-persona <hostpersona> <hostname> <iscsi_initiator_name> command.

# createhost -iscsi -persona 11 ESX2 iqn.1998-01.com.vmware:dl360g8-02-42b20fff

3. Verify that the host entry was created by using the 3PAR CLI showhost command.

# showhost

Id Name Persona ----------------WWN/iSCSI_Name---------------- Port1 ESX2 VMware iqn.1998-01.com.vmware:dl360g8-02-42b20fff 0:1:2

iqn.1998-01.com.vmware:dl360g8-02-42b20fff 1:1:2

4. To test the connection — create some temporary VVs and then export the VLUNs to thehost.

NOTE: See “Allocating Storage for Access by the ESX Host” (page 89) for completedetails on creating, exporting and discovering storage.

5. On the ESX iSCSI initiator host, perform a re-scan and then verify that the VLUNs werediscovered.

Create a Host Definition for iSCSI 25

Target Port Limits and Specifications for iSCSITo avoid overwhelming a target port and ensure continuous I/O operations, observe the followinglimitations on a target port:

• Follow the instructions for setting the maximum number of initiator connections supportedper array port, per array node pair, and per array as shown in the HPE 3PAR Support Matrixdocumentation at the SPOCK website (SPOCK Home→Other Hardware→3PAR).



51221 GbpsPCI-XF200, F400,T400, T800

iSCSIQLogicQLA4052C


iSCSIQLogicQLE8242

7000, 10000


iSCSIQLogicEP8324

8000, 20000

• The I/O queues are shared among the connected host server HBA ports on a first-come,first-served basis.

• When all queues are in use and a host HBA port tries to initiate I/O, it receives a target queuefull response from the 3PAR StoreServ Storage port. This condition can result in erratic I/Operformance on each host server. If this condition occurs, each host server should be throttledso that it cannot overrun the 3PAR StoreServ Storage port's queues when all host serversare delivering their maximum number of I/O requests.


3PAR Priority Optimization for iSCSIBeginning with 3PAR OS 3.1.2 MU2, the 3PAR Priority Optimization feature was introduced.This feature is an efficient and dynamic solution for managing server workloads and can be usedas an alternative to setting host I/O throttles. When using this feature, a storage administratorcan share storage resources effectively by enforcing quality of service limits on the array.SCSI queue full handling must be enabled on VMware hosts before enabling 3PAR PriorityOptimization on the array. For details, see “ESX/ESXi Handling SCSI Queue Full and BusyMessages from the 3PAR StoreServ Storage Array” (page 49). Other per target or per adapterthrottle settings might need to be adjusted in rare cases.For complete details of how to use 3PAR Priority Optimization (Quality of Service) on 3PARStoreServ Storage systems, see the HPE 3PAR Priority Optimization technical whitepaper:http://www.hpe.com/info/3PAR-Priority-Optimization




3PAR Persistent Ports for iSCSIBeginning with 3PAR OS 3.1.3:, the 3PAR Persistent Ports feature is supported for iSCSI.The 3PAR Persistent Ports (or virtual ports) feature minimizes I/O disruption on a 3PAR StoreServStorage in response to the following events:

• 3PAR OS firmware upgrade

• Node maintenance that requires the node to be taken offline (e.g., adding a new HBA)

• 3PAR node failure

• Array target ports being taken offline administrativelyEach iSCSI target storage array port has a partner array port automatically assigned by thesystem. Partner ports are assigned across array node pairs.3PAR Persistent Ports allows a 3PAR StoreServ Storage iSCSI port to assume the identity of afailed port while retaining its own identity. Where a given physical port assumes the identity ofits partner port, the assumed port is designated as a persistent port. Array port failover andfailback with 3PAR Persistent Ports is transparent to most host-based multipathing software,which can keep all of its I/O paths active.Beginning with 3PAROS 3.2.2, the Persistent Port feature for iSCSI has additional functionalityto minimize I/O disruption during an array port loss_sync event by supporting loss_sync recovery,automatic failover in case of a failure due to a loss of signal or a link going down between thearray and the switch.

NOTE:• Use of 3PAR Persistent Ports technology does not negate the need for properly installed,

configured, and maintained host multipathing software.• A key element for iSCSI connectivity is that partner ports must share the same IP network.


• Technical white paper HPE 3PAR StoreServ Persistent Ports (Hewlett PackardEnterprise document #F4AA4-4545ENW) at the Hewlett Packard Enterprise Support Centerwebsite.



3PAR Persistent Ports for iSCSI 27



4 Configuring the 3PAR StoreServ Storage for FCoESetting Up the FCoE Switch, FCoE Initiator, and FCoE target ports

Connect the ESX host FCoE initiator ports and the 3PAR StoreServ StorageFCoE target ports to the FCoE switches.

NOTE: FCoE switch VLANs, routing setup, and configuration is beyond the scope of this topic.For instructions on setting up VLANs and routing, refer to the manufacturer's guide for the switch.

1. CNA ports on 3PAR StoreServ 10000 and 7000 Storage arrays require a one timeconfiguration by using the controlport command. (3PAR T-Class, and F-Class arraysdo not require this one-time setting.)For example on a new FCoE config:

# showport

N:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol0:3:1 suspended config_wait - - cna -0:3:2 suspended config_wait - - cna -

# showport -i

N:S:P Brand Model Rev Firmware Serial HWType0:3:1 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3U4 CNA0:3:2 QLOGIC QLE8242 58 0.0.0.0 PCGLT0ARC1K3U4 CNA

2. If State=config_wait or Firmware=0.0.0.0, use the controlport config fcoe<n:s:p> command to configure. Use the showport and showport -i commands tocheck the configuration setting. For example:

# controlport config fcoe 0:3:1# controlport config fcoe 0:3:2# showport 0:3:1 0:3:2

N:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol Label Partner FailoverState0:3:1 target ready 2FF70002AC000121 20310002AC000121 host FCoE - - -0:3:2 target ready 2FF70002AC000121 20320002AC000121 free FCoE - - -

# showport -i 0:3:1 0:3:2

N:S:P Brand Model Rev Firmware Serial HWType0:3:1 QLOGIC QLE8242 58 4.11.122 PCGLT0ARC1K3U4 CNA0:3:2 QLOGIC QLE8242 58 4.11.122 PCGLT0ARC1K3U4 CNA

3. Check the current settings of the FCoE ports by using the showport -fcoe command.For example:

# showport -fcoe

N:S:P ENode_MAC_Address PFC_Mask0:3:1 00-02-AC-07-01-21 0x080:3:2 00-02-AC-06-01-21 0x00

28 Configuring the 3PAR StoreServ Storage for FCoE

Changing the configuration from iSCSI to FCoE1. Use the showport command.

# showport

0:3:1 target ready - 000E1E05BEE6 iscsi iSCSI - - -0:3:2 target ready - 000E1E05BEE2 iscsi iSCSI - - -

2. Turn off the iSCSI ports by using the controlport offline <node:slot:port>command:

# controlport offline 0:3:1# controlport offline 0:3:2# showport

0:3:1 target offline - 000E1E05BEE2 iscsi iSCSI0:3:2 target offline - 000E1E05BEE2 iscsi iSCSI

3. Change the topology to FCoE by using the controlport config fcoe<node:slot:port> and controlport rst <node:slot:port> commands:

# controlport config fcoe 0:3:1# controlport config fcoe 0:3:2# controlport rst 0:3:1# controlport rst 0:3:2

0:3:1 target offline - 000E1E05BEE2 iscsi iSCSI0:3:2 target offline - 000E1E05BEE2 iscsi iSCSI

# showport

0:3:1 target ready 2FF70002AC000121 20310002AC000121 host FCoE - - -0:3:2 target ready 2FF70002AC000121 20320002AC000121 free FCoE - - -

4. Check the current settings of the FCoE ports with the showport -fcoe command. Forexample:

# showport -fcoe

N:S:P ENode_MAC_Address PFC_Mask0:3:1 00-02-AC-07-01-21 0x080:3:2 00-02-AC-06-01-21 0x00

Setting Up the FCoE Switch, FCoE Initiator, and FCoE target ports 29

Create a Host Definition for FCoECreate a Host Definition for FCoE OverviewBefore connecting the ESX host to the 3PAR StoreServ Storage, a host definition must be created.The host definition must specify a valid host persona (host mode) for each 3PAR StoreServStorage port connected to a host HBA port either through a fabric or direct connection.

• With 3PAR OS 3.1.1 or earlier, ESX/ESXi uses the host Generic-Legacy Persona 6.With 3PAR OS 3.1.3 or later, host Generic-Legacy Persona 6 will not be supported for anyversion of VMware ESX deployed.With 3PAR OS 3.1.2 or later, Hewlett Packard Enterprise recommends migrating the ESXconfigurations to host VMware Persona 11. For details about migrating the ESX host fromhost Generic-Legacy Persona 6 to host VMware Persona 11, see “Host Persona Migration”(page 132).

NOTE: When changing an existing host persona from 6 to 11, a host reboot is requiredtor the change to take effect. This is an offline process. The host persona change shouldcoincide with changing the SATP rules on the host as well.

• Beginning with 3PAR OS 3.1.2, a second host VMware Persona 11 enables asymmetriclogical unit access (ALUA).

◦ Host VMware Persona 11 is required for FCoE end-to-end (FCoE target) configurations.


◦ To configure 3PAR Peer Persistence with Automatic Transparent Failover and ManualTransparent Switchover, see the HPE 3PAR Remote Copy Software User Guide in theHewlett Packard Enterprise Storage Information Library website.


• ESX/ESXi hosts performing 3PAR single volume peer motion are required to use 3PAR OS3.1.3 or later and host VMware Persona 11.

• For the appropriate host persona to use for ESX/ESXi hosts with 3PAR Remote Copyconfigurations, refer to the HPE 3PAR Remote Copy Software User Guide at the HewlettPackard Enterprise Storage Information Library website.

With both host Generic-Legacy Persona 6 and VMware Persona 11, see the appropriate topicsin this guide for host FCoE setup considerations.


NOTE: Beginning with 3PAR OS 3.2.1, an in-band device known as a PE LUN (ProtocolEndpoint LUN) is exposed to all hosts connected using host VMware Persona 11. The PE LUNis presented as part of the support for VMware vSphere 6.0 VVols. Depending on the hostHBA/CNA driver in use, the PE LUN might appear as a LUN number 256 in the host device list.In addition, a warning message related to the PE LUN might appear in the ESXi 6.0 vmkernellogs when using drivers that do not support VVols. However, the error does not interfere withnormal, non-VVol operation.

For complete details about using the controlport, createhost, and showhost commands,refer to the 3PAROSCommand Line Interface Reference or theHPE 3PARManagement ConsoleUser Guide at the Hewlett Packard Enterprise Storage Information Library website.


Create a Host Definition for FCoE 31



Creating a Host Definition for FCoE1. Display all available host personas:

# showhost -listpersona

2. To create host definitions, use the createhost command with the -persona option tospecify the persona and the host name.With 3PAR OS 3.1.1 or earlier:

# createhost -persona 6 ESXserver1 10000000C9724AB2 10000000C97244FE

With 3PAR OS 3.1.2 or later:

# createhost -persona 11 ESXserver1 10000000C9724AB2 10000000C97244FE

3. Verify that the host was created by using the showhost command.With 3PAR OS 3.1.1 or earlier, using Generic-Legacy Persona 6:

# showhost

Id Name Persona -WWN/iSCSI_Name- Port0 ESXserver1 Generic-legacy 10000000C9724AB2 --- 10000000C97244FE ---

With 3PAR OS 3.1.2 or later, using VMware Persona 11:

# showhost

Id Name Persona -WWN/iSCSI_Name- Port0 ESXserver2 VMware 100000051EC33E00 --- 100000051EC33E01 ---

Show the Persona_Name and Persona_Id relationship by using the showhost -personacommand.

# showhost -persona

Id Name Persona_Id Persona_Name Persona_Caps0 ESXserver1 6 Generic-legacy --1 Esxserver2 11 VMware SubLun, ALUA

NOTE:• If the host persona is not correctly set, use the sethost -persona <host number>

<hostname> command to correct the issue, where host number is 6 (for 3PAR OS3.1.1 or earlier) or 11 (recommended for 3PAR OS 3.1.2 or later).

• When changing the host persona to 11, reboot the ESX host. The host must be offlineor not connected in order to change the host persona from 6 to 11 or from 11 to 6.


Target Port Limits and Specifications for FCoETo avoid overwhelming a target port and to ensure continuous I/O operations, observe thefollowing limitations on a target port:• Follow the instructions for setting the maximum number of initiator connections supported

per array port, per array node pair, and per array as shown in the HPE 3PAR Support Matrixdocumentation at the SPOCK website (SPOCK Home→Other Hardware→3PAR).




FCoEQLogicQLE8242

7000, 10000


FCoEQLogicEP8324

8000, 20000

• The I/O queues are shared among the connected host HBA ports on a first-come, first-servedbasis.

• When all I/O buffer queues are consumed and a host HBA port tries to initiate I/O, it receivesa target queue full response from the 3PAR StoreServ Storage port. This condition can resultin erratic I/O performance on each host. If this condition occurs, each host should be throttledso that it cannot overrun the 3PAR StoreServ Storage port's queues when all hosts aredelivering their maximum number of I/O requests.

NOTE: When host ports can access multiple targets on fabric zones, the assigned targetnumber assigned by the host driver for each discovered target can change when the host isbooted and some targets are not present in the zone. This situation might change the devicenode access point for devices during a host restart. This issue can occur with any fabric-connectedstorage, and is not specific to the 3PAR StoreServ Storage.


3PAR Priority Optimization for FCoEBeginning with 3PAR OS 3.1.2 MU2, the 3PAR Priority Optimization feature was introduced.This feature is an efficient and dynamic solution for managing server workloads and can beutilized as an alternative to setting host I/O throttles. When using this feature, a storageadministrator can share storage resources effectively by enforcing quality of service limits on thearray.SCSI queue full handling must be enabled on VMware hosts before enabling 3PAR PriorityOptimization on the array. For details, see “ESX/ESXi Handling SCSI Queue Full and BusyMessages from the 3PAR StoreServ Storage Array” (page 49). Other per target or per adapterthrottle settings might need to be adjusted in rare cases.For complete details of how to use 3PAR Priority Optimization (Quality of Service) on 3PARStoreServ Storage systems, see the HPE 3PAR Priority Optimization technical whitepaper:http://www.hpe.com/info/3PAR-Priority-Optimization

Target Port Limits and Specifications for FCoE 33



3PAR Persistent Ports for FCoEThe 3PAR Persistent Ports (or virtual ports) feature minimizes I/O disruption during a 3PARStoreServ Storage online upgrade, node-down or cable pull event. Port shutdown or reset eventsdo not trigger this feature.Each FCoE target storage array port has a partner array port automatically assigned by thesystem. Partner ports are assigned across array node pairs.3PAR Persistent Ports allow a 3PAR StoreServ Storage FCoE port to assume the identity of afailed port while retaining its own identity. Where a given physical port assumes the identity ofits partner port, the assumed port is designated as a persistent port. Array port failover andfailback with 3PAR Persistent Ports is transparent to most host-based multipathing software,which can keep all of its I/O paths active.

NOTE: Use of 3PAR Persistent Ports technology does not negate the need for properly installed,configured, and maintained host multipathing software.


• Technical white paper HPE 3PAR StoreServ Persistent Ports (Hewlett PackardEnterprise document #F4AA4-4545ENW) at the Hewlett Packard Enterprise Support Centerwebsite.



3PAR Persistent Ports Setup and Connectivity Guidelines for FCoEBeginning with 3PAR OS 3.1.3:• The 3PAR Persistent Ports feature is supported for FCoE target ports (FCoE end-to-end

configurations).• The 3PAR Persistent Ports feature is enabled by default for 3PAR StoreServ Storage FCoE

ports during node-down events.Follow the specific cabling setup and connectivity guidelines for 3PAR Persistent Portsto function properly. Key elements for the 3PAR Persistent Ports feature setup andconnectivity are:

• 3PAR StoreServ Storage FCoE partner ports must be connected to the same FCoE network.

• The same CNA port on host-facing HBAs in the nodes of a node pair must be connected tothe same FCoE network, and preferably to different FCoE switches on the network.

• The FCoE network must support NPIV, and NPIV must be enabled.




5 Configure the Host for an FC ConnectionThis topic describes the procedures and considerations required to set up an ESX host tocommunicate with a 3PAR StoreServ Storage over an FC connection.

Installing the HBA and DriversBefore setting up the ESX host, make sure the host adapters are installed and operating properly.If necessary, see the documentation provided by the HBA vendor for instructions.Drivers for VMware supported HBAs are included as part of the ESX OS installation packagesupplied by VMware. Updates and patches for the HBA drivers are available through VMwaresupport.For Brocade FC HBA, the default Path TOV (Time-out Value) parameter is set to 30 seconds.Hewlett Packard Enterprise recommends changing this value to 14 seconds with VMware NativeMultipathing Plug-in (NMP). To change the value of this parameter, use the Brocade BCUcommand line utility. For more information, see the VMware website.To download Brocade HBA drivers, firmware, and the BCU utility, see the QLogic website.Specific Brocade drivers are named with a BR prefix, such as BR-xxx HBA model.To display the list of adapter ports:

# esxcli brocade bcu --command="port --list"

---------------------------------------------------------------------------Port# FN Type PWWN/MAC FC Addr/ Media State Spd

Eth dev---------------------------------------------------------------------------1/0 - fc 10:00:00:05:1e:dc:f3:2f 091e00 sw Linkup 8G

0 fc 10:00:00:05:1e:dc:f3:2f 091e00 sw Linkup 8G1/1 - fc 10:00:00:05:1e:dc:f3:30 673000 sw Linkup 8G

1 fc 10:00:00:05:1e:dc:f3:30 673000 sw Linkup 8G

To query a port number from the previous output:

# esxcli brocade bcu --command="vhba --query 1/0"

PCI Function Index : 1/0/0Firmware Ver : 3.0.0.0Port type : FCBandwidth : 8 GbpsIOC state : operationalPWWN : 10:00:00:05:1e:dc:f3:2fNWWN : 20:00:00:05:1e:dc:f3:2fPath TOV : 30 secondsPortlog : EnabledIO Profile : OffInterrupt coalescing : onInterrupt delay : 0 usInterrupt latency : 0 us

Installing the HBA and Drivers 35

To change Path TOV value (repeat for all ports) follow the example below. This command canbe included in ESX host startup so that it runs automatically.

# esxcli brocade bcu --command="fcpim --pathtov 1/0 14"

path timeout is set to 14

To query a port number after a change was made, follow the example below:

# esxcli brocade bcu --command="vhba --query 1/0"

PCI Function Index : 1/0/0Firmware Ver : 3.0.0.0Port type : FCBandwidth : 8 GbpsIOC state : operationalPWWN : 10:00:00:05:1e:dc:f3:2fNWWN : 20:00:00:05:1e:dc:f3:2fPath TOV : 14 secondsPortlog : EnabledIO Profile : OffInterrupt coalescing : onInterrupt delay : 0 usInterrupt latency : 0 us

More informationhttp://www.vmware.comhttp://www.qlogic.com

Install Virtual Machine Guest OSThe VMware ESX host documentation provides a list of recommended VM (virtual machine)guest OSs with their installation and setup as VMs. See the VMware ESX host documentationfor information about setting up the VM configuration.

CAUTION:• In VMware KB 51306, VMware identifies a problem with RHEL 5 (GA), RHEL 4 U4,

RHEL 4 U3, SLES 10 (GA), and SLES 9 SP3 guest OSs. Their file systems might becomeread-only in the event of busy I/O retry or path failover of the ESX host’s SAN or iSCSIstorage. For more information, refer to KB 51306 at the VMware Knowledge Base website.Hewlett Packard Enterprise does not recommend and does not support RHEL 5 (GA),RHEL 4 U4, RHEL 4 U3, SLES 10 (GA), and SLES 9 SP3 as guest OSs for VMs on VMwareESX hosts attached to 3PAR StoreServ Storage systems.

• Deployment of the NPIV (N-Port ID Virtualization) feature is not recommended. For NPIVfeature limitations, refer to KB 2116686 at the VMware Knowledge Base website.

• VMware and Hewlett Packard Enterprise recommend the LSI logic adapter emulation forWindows 2003 Servers. The LSI Logic adapter is also the default option for Windows 2003when creating a new VM. Hewlett Packard Enterprise testing has noted a high incidence ofWindows 2003 VM failures during an ESX multipath failover/failback event when the BUSLogic adapter is used with Windows 2003 VMs.

• Hewlett Packard Enterprise testing found that the SCSI timeout value for VM guest OSsshould be 60 seconds to successfully manage path failovers at the ESX layer. Most guestOSs supported by VMware have a default SCSI timeout value of 60 seconds, but this valueshould be checked and then verified for each guest OS installation. In particular, Red Hat

36 Configure the Host for an FC Connection

http://www.vmware.com

http://www.qlogic.com

4.x guest OSs should have the SCSI timeout value changed from the default 30 seconds to60 seconds.Use the command line to set the SCSI timeout on all SCSI devices presented to a Red Hat4.x VM to 60 seconds:find /sys -name timeout | grep "host.*target.*timeout" | xargs -n1 echo "echo 60 >"|sh

Add this line in the /etc/rc.local file of the Red Hat 4.x guest OS to maintain the timeoutchange with a VM reboot.Example of a modified /etc/rc.local file:

# cat /etc/rc.local

#!/bin/sh## This script will be executed *after* all the other init scripts.# You can put your own initialization stuff in here if you don't# want to do the full Sys V style init stuff.

find /sys -name timeout | grep "host.*target.*timeout" | xargs -n 1 echo "echo 60 >"|shtouch /var/lock/subsys/local

On Windows Server, check the TimeOutValue by using the following command:

C:\ reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\disk /v TimeOutValue

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\disk TimeOutValue REG_DWORD 0x3c

More informationhttp://kb.vmware.com

Install Virtual Machine Guest OS 37


Multipath Failover Considerations and I/O Load BalancingNOTE: This topic about multipathing and configuring to Round Robin policy applies to allconnectivity types: FC, FCoE, and iSCSI.

With VMware ESX 3.0 - 3.5, failover with multipath support is included to maintain a constantconnection between the ESX host and the 3PAR StoreServ Storage array. VMware terms thismultipath support with a choice of two path policies called Fixed or MRU. Beginning withESX 4.0, a third path policy choice of round robin is available. The path policies can bemodified on a per-3PAR StoreServ Storage-volume (LUN) basis by right-clicking the device listingand selecting the Properties function from the vSphere client menu. A pop-up window allowsfor managing paths, so that the properties of the paths to the volume selected can be modified.When using this control, select the path policy, and specify which path is the active preferredpath to a volume on the storage array, or which path is the standby path within the Fixed pathpolicy scheme. Additionally, paths can be disabled to prevent any traffic over a specific path toa volume on the storage array.The vSphere client GUI allows for settings to be changed only on a volume-by-volume(LUN-by-LUN) basis. The GUI is appropriate and preferred for use in managing I/O paths withinthe FIXED path policy scheme. For procedures on implementing and configuring the round-robinpath policy on ESX/ESXi 4.0 and later with a 3PAR StoreServ Storage, see “Configure RoundRobin Multipathing on ESX 4.x or later for FC” (page 40).

• A path policy of "round-robin" is the preferred multipath implementation for ESX/ESXi 4.0and later. For procedures on implementing and configuring the round-robin path policy onESX/ESXi 4.0 and later with a 3PAR StoreServ Storage, see “Configure Round RobinMultipathing on ESX 4.x or later for FC” (page 40).

• A path policy of Fixed and the preferred/active paths manually set to balance I/O load evenlyacross all paths is the preferred multipath implementation for ESX 3.0 - 3.5.

◦ In the event the active path fails or is disabled either at the fabric switch, or on thestorage array, all ESX host I/O to the storage array continues by failing over to a standbypath. When the ESX host detects that the preferred path is recovered or is enabled, I/Ofrom the ESX host resumes on the preferred path -- as long as a preferred path policypath was set.

◦ I/O from the ESX host should be manually distributed or balanced when two or morepaths exist to more than one 3PAR StoreServ Storage volume on the storage array.Manually balancing the loads across available paths might improve I/O performance.This path load balancing to the storage array is dependent on the number of I/Os targetedfor specific volumes on the storage array. Tuning I/Os to specific volumes on specificpaths to the storage array varies from configuration to configuration and is totallydependent on the workload from the ESX host and the VMs to the devices on the storagearray.

“vSphere Client” (page 39) shows a LUN with five I/O paths in a FIXED I/O policy scheme. Thepath marked Active (I/O) * is the preferred path, and is the path to which all I/O is currentlyassigned for the given LUN. The other paths are listed as active, but are in standby mode. Thepaths in active standby mode will not be used for I/O traffic for this LUN unless the preferred pathfails.


Figure 1 vSphere Client

• A path policy of MRU (most recently used) does not maintain or reinstate balancing of I/Oload after a failover/failback multipath event. This could leave I/O in an unplanned andunbalanced state which might yield significant I/O performance issues. Hewlett PackardEnterprise does not recommend the Implementation of an MRU path policy.

NOTE: If I/O is active to a LUN and an attempt is made to modify the path policy, a failure canoccur, indicating:"error during the configuration of the host: sysinfoException;Status=Busy: Message=Unable to Set".

If this problem occurs while attempting to change the path policy, reduce the I/Os to that LUNand then try making the change again.

For additional information on this topic, refer to the topic on "Multipathing" in the SANConfigurationGuide at the VMware website.

More informationhttp://www.vmware.com

Multipath Failover Considerations and I/O Load Balancing 39


Configure Round Robin Multipathing on ESX 4.x or later for FCWith ESX 4.0 or later, VMware supports a round-robin I/O path policy for active/active storagearrays such as 3PAR StoreServ Storage. A round-robin I/O path policy is the preferredconfiguration for ESX 4.0 and later; however, this path policy is not enabled by default for 3PARdevices.

CAUTION: When running Windows Server 2012 or Windows Server 2008 VM Clusterwith RDM shared LUNs on hosts with ESXi 5.5 or earlier, individually change these specificRDM LUNs from Round Robin policy to FIXED or MRU path policy.

“LUN Set to Round Robin” (page 40), shows output from an FC configuration, a LUN with a paththat was set to Round Robin (VMware).

NOTE: Each path status is shown as Active (I/O). The path status for an iSCSI configurationwould be the same.

Figure 2 LUN Set to Round Robin


Managing a round robin I/O path policy scheme through the vSphere client GUI for a large networkcan be cumbersome and challenging to maintain because the policy must be specified for eachLUN individually and then updated when new devices are added. Alternatively, VMware providesa way for the server administrator to use ESX CLI, vCLI, or vSphere Management Assistant(vMA) commands to manage I/O path policy for storage devices on a per-host basis by usingparameters defined in a set of native ESX/ESXi storage plug-ins.The VMware native multipathing has two important plug-ins:• The Storage Array Type Plug-in (SATP) — handles path failover and monitors path health.

• The Path-selection Plug-in (PSP) — chooses the best path and routes I/O requests for aspecific logical device, that is, PSP defines the path policy.

The correct ESX/ESXi host Storage Array Type Plug-in (SATP) to be used is related to the 3PARarray host persona:

• When 3PAR host Generic-Legacy Persona 6 is the host persona in use with an ESX/ESXihost, use the SATP VMW_SATP_DEFAULT_AA.

• When 3PAR host VMware Persona 11 is the host persona in use with an ESX/ESXi host,use the SATP VMW_SATP_ALUA.

For ESX/ESXi 4.0 versions (4.0 GA through all 4.0 updates), the default SATP rules must beedited in order to automatically achieve a round robin I/O path policy for storage devices.Beginning with ESX/ESXi 4.1, additional custom SATP rules can be created that target SATP/PSPto specific vendors while leaving the default SATP rules unmodified. The custom SATP can beused to automatically achieve a round robin I/O path policy for storage devices.


Configure ESX/ESXi Multipathing for Round Robin via SATP PSPAs part of the PSP Round-Robin configuration, the value of IOPS can be specified. IOPS is thenumber of IO operations scheduled for each path during path changes within the Round-Robinpath selection scheme. The default IOPS value is 1000. Hewlett Packard Enterprise recommendsIOPS=1 as an initial value and starting point for further optimization of IO throughput with PSPRound-Robin. With the exception of ESX/ESXi 4.0 versions, it is preferable to set the IOPS valuewithin a SATP custom rule.

CAUTION: VMware specifically warns not to directly edit the esx.conf file.

NOTE:• The setting of IOPs=1 is a new recommended initial value as documented in the technical

white paper HPE 3PAR StoreServ Storage and VMware vSphere 5 Best Practice (HewlettPackard Enterprise document #4AA4–3286ENW). Change this setting to suit the demandsof various workloads.

• SATP rule changes cannot be affected through vSphere GUI.

• SATP rule changes through ESX CLI commands populate the esx.conf file.

• A custom SATP rule is an additional SATP rule that either modifies or redefines parametersof an existing SATP default rule, defines the targeted devices affected, and has a uniquecustom rule name.

• A custom SATP rule cannot be changed or edited. A custom SATP rule must be removedbefore creating a new one with the changes added to affect a change in the parameters ofthe custom rule.

• SATP and PSP creation, changes, additions, or removals take effect for any new devicespresented afterward without the need for server reboot.

• The host must be rebooted for SATP rule creation, changes, additions, or removals to takeeffect on existing, previously presented devices.

• Path policy changes done on an individual device basis, either done via VMware vCenteror ESX CLI command, supersede the PSP path policy defined in a SATP rule, and suchpath policy changes to individual devices are maintained through host reboots.

• Valid PSP for SATP VMW_SATP_DEFAULT_AA rules are:

◦ VMW_PSP_RR

◦ VMW_PSP_FIXED

◦ VMW_PSP_MRU

VMW_PSP_RR is preferred.

• Valid PSP for SATP VMW_SATP_ALUA rules are:

◦ VMW_PSP_RR

◦ VMW_PSP_MRU

VMW_PSP_FIXED is not a valid PSP to be defined within an ALUA SATP rule. VMW_PSP_RRis preferred.


Changing from 3PAR host Generic-Legacy Persona 6 to host VMware Persona 11 or viceversa:

• A change from host Generic-Legacy Persona 6 to VMware Persona 11, or from host VMwarePersona 11 to Generic-Legacy Persona 6, the 3PAR OS requires either taking the affectedarray ports offline, or disconnecting the host for which the persona is being changed.

• For existing devices targeted in a custom SATP rule to be claimed by the rule, the ESX/ESXiOS requires a host reboot.

Hewlett Packard Enterprise recommends using the following procedure to change from hostGeneric-Legacy Persona 6 to VMware Persona 11 or from host persona VMware Persona 11 toGeneric-Legacy Persona 6:1. Stop all host I/O and apply the necessary SATP changes (create custom SATP rule and/or

modify default SATP rule PSP defaults) to the ESX/ESXi host.2. Shut down the host.3. Change the host persona on the array.4. Boot the host.5. Verify that the target devices have been claimed properly by the SATP rule.With ESX/ESXi 4.0 GA - 4.0 MUx:

NOTE:• Although ESX 4.0 GA - 4.0 MUx supports custom SATP rules, the -P options for setting

PSP (path policy) within the custom rule are not supported. PSP must be defined within thedefault SATP rules.

• ESX 4.0 GA - 4.0 MUx has a known issue where no attempt should be made to changeiops from its default value of 1000 (iops = 1000). If the iops value is changed, at thenext host reboot, an invalid value for iops is retrieved and iops become unpredictable.

• If the custom rule is not created for ALUA, 3PARdata VVs are claimed by theVMW_SATP_DEFAULT_AA SATP rule even though the array host persona is 11/VMware(ALUA compliant array port presentation).

3PAR SATP rules for use with host Generic-Legacy Persona 6 (Active-Active array portpresentation):A “custom” SATP rule is not used. The PSP (path policy) is changed on the default active-activeSATP rule. The default multipath policy for VMW_SATP_DEFAULT_AA is VMW_PSP_FIXED (Fixedpath). The default is changed to the preferred PSP (path policy) of round-robin.

# esxcli nmp satp setdefaultpsp -s VMW_SATP_DEFAULT_AA -P VMW_PSP_RR

3PAR SATP rules for use with host VMware Persona 11 (ALUA compliant array portpresentation):

# esxcli nmp satp setdefaultpsp -s VMW_SATP_ALUA -P VMW_PSP_RR

# esxcli nmp satp addrule -s "VMW_SATP_ALUA" -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PAR Custom iSCSI/FC/FCoE ALUARule"

To remove the above ALUA custom SATP rule:

# esxcli nmp satp deleterule -s "VMW_SATP_ALUA" -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PAR Custom iSCSI/FC/FCoE ALUARule"


CAUTION: The procedure for changing the default SATP rules to use the round robin I/Omultipathing policy is intended to apply only to VMware hosts using 3PAR StoreServ StorageLUNs. If the host shares storage from other vendors, before making any I/O policy changes,consider the effect that changing the default rules will have on the storage environment as awhole.A change of the default PSP for a given SATP affects all storage devices (FC, FCoE, iSCSI) thatuse the same default SATP rule. If a host shares multiple storage vendors in addition to a 3PARStoreServ Storage, and the other connected storage does not support active/active round robinmultipathing using the same SATP rule, such as VMW_SATP_DEFAULT_AA orVMW_DEFAULT_ALUA, then its multipathing is also affected.If the other storage uses a different SATP of its own, then SATP VMW_SATP_DEFAULT_AAmapping should be changed to VMW_PSP_RR to take advantage of round-robin multipathing.Check the SATP-PSP relationship of a given device for ESX 4.0 by using the esxcli nmpdevice list or esxcli nmp device list -d <device id> command.For example, If the 3PAR StoreServ Storage and storage X are connected to the same hostusing VMW_SATP_DEFAULT_AA, and if storage X does not have its own SATP, then it mightcause a problem if storage X does not support round-robin multipathing. If the 3PAR StoreServStorage and storage Y are sharing the same host, and if storage Y has its own SATP VMW_SATP_Yand Hewlett Packard Enterprise uses VMW_SATP_DEFAULT_AA, then there will be no conflict,and the change can be made.

With ESX/ESXi 4.1 GA - 4.1 MUx:3PAR custom SATP rule for use with host Generic-Legacy Persona 6 (Active-Active arrayport presentation):

# esxcli nmp satp addrule -s "VMW_SATP_DEFAULT_AA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_off" -V "3PARdata" -M "VV" -e "HP3PAR Custom iSCSI/FC/FCoE Rule"

To remove the above active-active custom SATP rule:

# esxcli nmp satp deleterule -s "VMW_SATP_DEFAULT_AA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_off" -V "3PARdata" -M "VV" -e"HP 3PAR Custom iSCSI/FC/FCoE Rule"

3PAR custom SATP rule for use with host VMware Persona 11 (ALUA compliant array portpresentation):

# esxcli nmp satp addrule -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PARCustom iSCSI/FC/FCoE ALUA Rule"


# esxcli nmp satp deleterule -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PARCustom iSCSI/FC/FCoE ALUA Rule"


With ESXi 5.x and ESXi 6.0:3PAR custom SATP rule for use with host Generic-Legacy Persona 6 (Active-Active arrayport presentation):

# esxcli storage nmp satp rule add -s "VMW_SATP_DEFAULT_AA" -P "VMW_PSP_RR" -O iops=1 –c "tpgs_off" -V "3PARdata" -M "VV"-e "HP 3PAR Custom iSCSI/FC/FCoE Rule"

To remove the above Active-Active custom SATP rule:

# esxcli storage nmp satp rule remove -s "VMW_SATP_DEFAULT_AA" -P "VMW_PSP_RR" -O iops=1 –c "tpgs_off" -V "3PARdata" -M"VV" -e "HP 3PAR Custom iSCSI/FC/FCoE Rule"

3PAR custom SATP rule for use with host VMware Persona 11 (ALUA compliant array portpresentation):

# esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP3PAR Custom iSCSI/FC/FCoE ALUA Rule"


# esxcli storage nmp satp rule remove -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e"HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule"


SATP Info Commands

Default SATP Rules and Their Current Default PSPList default SATP rules and their current default PSP (path policy) by using the commands in thefollowing examples:ESXi 5.x and ESXi 6.0 example:

#esxcli storage nmp satp list

Name Default PSP Description-------------------- ------------- -------------------------------------------------------VMW_SATP_ALUA VMW_PSP_MRU Supports non-specific arrays that use the ALUA protocolVMW_SATP_MSA VMW_PSP_MRU Placeholder (plugin not loaded)VMW_SATP_Default_AP VMW_PSP_MRU Placeholder (plugin not loaded)VMW_SATP_SVC VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_BQL VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_INV VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_EVA VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_ALUA_CX VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_SYMM VMW_PSP_FIXED Placeholder (plugin not loaded)VMW_SATP_CX VMW_PSP_MRU Placeholder (plugin not loaded)VMW_SATP_LSI VMW_PSP_MRU Placeholder (plugin not loaded)VMW_SATP_Default_AA VMW_PSP_FIXED Supports non-specific active/active arraysVMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices

ESX/ESXi 4.x example:

#esxcli storage nmp satp list

Name Default PSP DescriptionVMW_SATP_ALUA_CX VMW_PSP_FIXED Supports EMC CX that use the ALUA protocolVMW_SATP_SVC VMW_PSP_FIXED Supports IBM SVCVMW_SATP_MSA VMW_PSP_MRU Supports HP MSAVMW_SATP_EQL VMW_PSP_FIXED Supports EqualLogic arraysVMW_SATP_INV VMW_PSP_FIXED Supports EMC InvistaVMW_SATP_SYMM VMW_PSP_FIXED Supports EMC SymmetrixVMW_SATP_LSI VMW_PSP_MRU Supports LSI and other arrays compatible with the SIS 6.10 in non-AVT modeVMW_SATP_EVA VMW_PSP_FIXED Supports HP EVAVMW_SATP_DEFAULT_AP VMW_PSP_FIXED Supports non-specific active/passive arraysVMW_SATP_CX VMW_PSP_MRU Supports EMC CX that do not use the ALUA protocolVMW_SATP_ALUA VMW_PSP_RR Supports non-specific arrays that use the ALUA protocolVMW_SATP_Default_AA VMW_PSP_RR Supports non-specific active/active arraysVMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices


SATP Custom Rules and Associated Defined ParametersTo list SATP custom rules and associated defined parameters, use the commands in the followingexamples:ESXi 5.x and ESXi 6.0 example:For VMware Persona 11:

# esxcli storage nmp satp rule list | grep -i 3par

VMW_SATP_ALUA 3PARdata VV user tpgs_onVMW_PSP_RR iops=1 HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule

For Generic-Legacy Persona 6:


VMW_SATP_DEFAULT_AA 3PARdata VV user tpgs_offVMW_PSP_RR iops=1 HP 3PAR Custom iSCSI/FC/FCoE Rule

ESX/ESXi 4.x example:

# esxcli nmp satp listrules | grep -i 3par

VMW_SATP_ALUA 3PARdata VVtpgs_on HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule

Showing Device InformationTo show device information, use the commands in the following examples:ESXi 5.x and ESXi 6.0 example:

# esxcli storage nmp device list

naa.50002ac0000a0124Device Display Name: 3PARdata iSCSI Disk (naa.50002ac0000a0124)Storage Array Type: VMW_SATP_ALUAStorage Array Type Device Config: {implicit_support=on;explicit_support=on;

explicit_allow=on;alua_followover=on;{TPG_id=256,TPG_state=AO}}Path Selection Policy: VMW_PSP_RRPath Selection Policy Device Config: {policy=rr,iops=1,bytes=10485760,useANO=0;lastPathIndex=1:

NumIOsPending=0,numBytesPending=0}Path Selection Policy Device Custom Config:Working Paths: vmhba3:C0:T1:L73, vmhba2:C0:T0:L73, vmhba2:C0:T1:L73, vmhba3:C0:T0:L73

ESX/ESXi 4.x example:The command is the same with ESX/ESXi 4.x. The output shown is for ESX 4.0:

# esxcli nmp device list

naa.50002ac000b40125Device Display Name: 3PARdata Fibre Channel Disk (naa.50002ac000b40125)Storage Array Type: VMW_SATP_DEFAULT_AAStorage Array Type Device Config:Path Selection Policy: VMW_PSP_RRPath Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0;lastPathIndex=3:

NumIOsPending=0,numBytesPending=0}Working Paths: vmhba5:C0:T0:L25, vmhba5:C0:T1:L25, vmhba4:C0:T0:L25, vmhba4:C0:T1:L25

With ESX 4.1, the iops will be 1 for the device list output shown above.


Script Alternative for Path Policy Changes on Storage Devices without a Host RebootIf a reboot of the ESX/ESXi host to affect path policy changes through SATP on a large numberof existing, previously presented storage devices is not desirable, the path policy changes on abatch of LUNs can be made by scripting ESX CLI commands.

Changing Path Policy on a Batch of LUNs Using CLI Commands

NOTE: If I/O is active to a LUN and an attempt is made to modify the path policy, a failure canoccur. If this problem occurs during an attempt to change the path policy, reduce the I/Os to thatLUN and then make the change.

error during the configuration of the host: sysinfoException;

Status=Busy: Message=Unable to Set

Create a script that uses the following commands:1. List all the 3PAR devices present on the host:

ESXi 5.x and ESXi 6.0:

# esxcli storage nmp device list | grep -i naa.50002ac | grep -v Device

naa.50002ac0005800acnaa.50002ac003b800acnaa.50002ac0039300ac

ESX/ESXi 4.x:

# esxcli nmp device list | grep -i naa.50002ac | grep -v Device

naa.50002ac0005800acnaa.50002ac003b800acnaa.50002ac0039300ac

2. Change the I/O path policy to round robin for each device identified in the previous output:ESXi 5.x and ESXi 6.0:

# esxcli storage nmp device set -d naa.50002ac0005800ac -P VMW_PSP_RR

ESX/ESXi 4.x:

# esxcli nmp device setpolicy -d naa.50002ac0005800ac -P VMW_PSP_RR

3. Verify that the change has been made.ESXi 5.x and ESXi 6.0:

# esxcli storage nmp device list -d naa.50002ac0005800ac

ESX/ESXi 4.x:

# esxcli nmp device list -d naa.50002ac0005800ac


Performance Considerations for Multiple Host ConfigurationsThe information in this topic should be considered when using multiple ESX hosts, (or other hostsin conjunction with ESX hosts), that are connected in a fan-in configuration to 3PAR StoreServStorage ports.

ESX/ESXi Handling SCSI Queue Full and Busy Messages from the 3PAR StoreServStorage Array

VMware ESX Release ESX 4.x, ESXi 5.0 and 5.0 Updates, ESXi 5.5 and 5.5 Updates, and ESXi6.0

ESX 4.0, 4.1 and ESXi 5.0, 5.5, 6.0:With ESX 4.0 GA, ESX 4.1 (with all ESX 4.x updates), ESXi 5.0 (with all updates), ESXi 5.5(with all updates), and ESXi 6.0, an algorithm was added that allows ESX to respond to QueueFull and Busy SCSI messages from the storage array in accordance with industry standards.The Queue Full or Busy response by ESX is to throttle I/O for a period of time, this helps toprevent over driving of the 3PAR StoreServ Storage ports.

NOTE: This feature is required to be enabled as part of an ESX/ESXi - 3PAR StoreServ Storagedeployment in a multi-host environment.

The Queue Full and Busy LUN-throttling algorithm is disabled by default.With the algorithm enabled, no additional I/O throttling scheme is necessary. For additionalinformation regarding the ESX Queue Full/Busy response algorithm, refer to KB 1008113 atthe VMware Knowledge Base website.

NOTE: A setting of QFullSampleSize = 0 will turn the throttling algorithm off and the host willnot adjust I/O load to Queue Full or Busy responses from an array.

Enabling the Queue Full and Busy LUN-throttling algorithm1. From the vSphere client, select the ESX host. In the Configuration tab, select Advanced

Settings for software, and then select Disk.2. Scroll to find and then adjust the following HPE 3PAR recommended settings:

QFullSampleSize = 32QFullThreshold = 4

Performance Considerations for Multiple Host Configurations 49

VMware ESXi Release 5.1:The Advanced Settings parameters QFullSampleSize and QFullThreshold are required toenable the adaptive queue-depth algorithm.With versions earlier than ESXi 5.1, these parameters are set globally; that is, they are set onall devices seen by the ESXi host.With ESXi 5.1, however, these parameters are set in a more granular fashion, on a per-devicebasis.VMware patch ESXi510-201212001, dated 12/20/2012 (KB 2035775), restores the ability to setthe values of these parameters globally. See the VMware Knowledge Base website.To set the parameters globally, install the patch and follow the instructions in “VMware ESXRelease ESX 4.x, ESXi 5.0 and 5.0 Updates, ESXi 5.5 and 5.5 Updates, and ESXi 6.0” (page 49).Also, use the esxcli command to set these values on a per-device basis. If both options (theesxcli and the advanced parameters) are used, the per-device values take precedence.Set QFullSampleSize and QFullThreshold on a per-device level by using the esxclicommand:

# esxcli storage core device set --device device_name -q Q -s S

The settings do not require a reboot to take effect and are persistent across reboots.Retrieve the values for a device by using the corresponding list command:

# esxcli storage core device list

The command supports an optional --device parameter:

# esxcli storage core device list --device device_name




ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature ConsiderationsBeginning with ESX/ESXi 4.1, new features related to storage I/O control and integration withstorage arrays are introduced . With ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 3PAR StoreServStorage configurations, Hewlett Packard Enterprise recommends using SIOC and vStorageAPIs for Array Integration (VAAI).

NOTE: This topic about VAAI and new features applies to all connectivity types: FC, FCoEand iSCSI.

Storage I/O ControlThe SIOC feature allows a new level of monitoring and control of I/O from individual VMs to a3PAR StoreServ Storage array at the datastore level and across ESX/ESXi hosts in a VMwarecluster.For more information about the SIOC feature and considerations for its deployment, refer to theVMware documentation regarding the Storage I/O Control feature and vSphere ResourceManagement.

VAAI (vStorage APIs for Array Integration)In partnership with VMware, Hewlett Packard Enterprise has developed an ESX/ESXi 4.1 plug-inthat enables a new set of SCSI commands to be used by ESX/ESXi 4.1 in conjunction with 3PARStoreServ Storage. VMware refers to this newly incorporated set of SCSI commands as the"primitives".ESX extensions that make use of these primitives are collectively referred to as vStorage APIsfor Array Integration (VAAI). The VMware primitives allow an ESX/ESXi host to send VM operationsto storage hardware at a meta level instead of at the traditional data level. This reduces operationallatency and traffic on the FC fabric/iSCSI network. Some of these primitives allow the storagehardware to participate in block allocation and de-allocation functions for VMs. These primitivesare also known as hardware “offloads”.A brief description of the "primitives":

• Full Copy (XCOPY) enables the storage array to make full copies of data within the arraywithout having to have the ESX host read and write the data. This off loads some data copyprocesses to the storage array.

• Block Zeroing (WRITE-SAME) enables the storage array to zero-out a large number ofblocks within the array without having to have the ESX host write the zeros as data andhelps expedite the provisioning of VMs. This off loads some of the file space zeroing functionsto the storage array.

• Hardware Assisted Locking (ATS) provides an alternative to SCSI reservations as a meansto protect the metadata for VMFS cluster file systems and helps improve the scalability oflarge ESX host farms sharing a datastore.

ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature Considerations 51

3PAR VAAI Plug-in 1.1.1 for ESX/ESXi 4.1With 3PAR OS version 3.1.1 or later and with ESX/ESXi 4.1, support for VMware VAAIfunctionality is available by installing the 3PAR VAAI Plug-in 1.1.1.Beginning with 3PAR OS 3.1.2 and using 16-byte volume WWNs, there is a 3PAR VAAIplug-in v1.1.1 limitation in which VAAI Full Copy (XCOPY) does not function, and data copyprocesses are not offloaded to the array.For more information about VMware VAAI, the 3PAR VAAI Plug-in for ESX/ESXi 4.1 installationpackage, and the HPE 3PAR VAAI Plug-in 1.1.1 for VMware vSphere 4.1 User's Guide, see theHewlett Packard Enterprise Software Depot website.

More informationhttp://www.hpe.com/support/softwaredepot

3PAR VAAI Plug-in 2.2.0 for ESXi 5.x and ESXi 6.0With 3PAR OS 3.1.1 or later, do not install the 3PAR VAAI Plug-in 2.2.0 on the ESXi 5.x. TheVAAI primitives are handled by the default T10 VMware plug-in and do not require the 3PARVAAI plug-in.The following table summarizes the 3PAR VAAI Plug-in installation requirements.

Table 1 3PAR VAAI Plug-in Installation Requirements

ESXi 5.x, ESXi 6.0ESX 4.1VMware ESXFeatures/Primitives

Supported. Does not require 3PAR VAAIplug-in (supported by Standard T10 ESXplug-in).

Requires 3PAR VAAI 1.1.1 plug-in

ATS

XCOPY

WRITE_SAME


http://www.hpe.com/support/softwaredepot

UNMAP (Space Reclaim) Storage Hardware Support for ESXi 5.x or ESXi 6.0With 3PAR OS 3.1.1 or later and beginning with ESXi 5.0 Update 1 with default VMwareT10 VAAI plug-in and ESXi 6.0, the UNMAP storage primitive for space reclaim is supported.Installation of the 3PAR VAAI plug-in is not required.

NOTE: To avoid possible issues described in VMware KB 2007427 and KB 2014849, automaticVAAI Thin Provisioning Block Space Reclamation (UNMAP) should be disabled on ESXi 5.0 GA.Refer to the KB articles at the VMware Knowledge Base website.

ESXi 5.0 Update 1 and later includes an updated version of vmkfstools that provides an option[-y] for sending the UNMAP command regardless of the ESXi host’s global setting.Use the [-y] option:

# cd /vmfs/volumes/<volune-name>

vmkfstools -y <percentage of deleted block to reclaim>

NOTE: The vmkfstools -y option does not work in ESXi 5.0 GA.

ESXi 5.5 introduces a new command in the esxcli namespace that allows deleted blocks tobe reclaimed on thin provisioned LUNs that support the VAAI UNMAP primitive.

# esxcli storage vmfs unmap -l <datastore name>

The vmkfstools -y command is deprecated in ESXi 5.5. See VMware KB 2057513 for moreinformation.UNMAP also frees up space if files are deleted on UNMAP-supported VMs such as Red HatEnterprise 6, as long as it is a RDM LUN on a TPVV storage volume—for example, for RDMvolumes on a Red Hat VM using the ext4 filesystem and mounted using the discard option.

# mount —t ext4 —o discard /dev/sda2 /mnt

This causes Red Hat 6 VM to run the UNMAP command and release space back on the array forany deletes in that ext4 file system.




Out-of-Space Condition for ESX 4.1, ESXi 5.x, or ESXi 6.0With ESX 4.1, ESXi 5.x, or ESXi 6.0 and 3PAR OS 3.1.1 and later, the out-of-space condition,aka "VM STUN", is implemented. This OS feature does not depend on the VAAI plug-in andapplies to TPVV volume types.When the TPVV cannot allocate additional storage space or cannot grow because the storagesystem is out of disk space, it sends a Check condition with "DATA PROTECT" sense key errorand additional sense "SPACE ALLOCATION FAILED WRITE PROTECT". As a result, ESXpauses the VM and displays an 'Out of Space' message to the user in the Summary tab of theVM on vSphere, with the options of Retry or Cancel. In the pause-VM condition, read requestsand rewrites to the allocated LUN blocks are allowed, but writing to a new space is not allowed.Ping, telnet and ssh requests to the VM are not honored. The storage administrator must addadditional disk space or use storage vMotion to migrate other unaffected VMs from the LUN.After additional disk space is added, use the Retry option on the warning message to bring theVM back to the read-write state. If the Cancel option is selected, the VM reboots.In the following example, a 3PAR StoreServ Storage TPVV was created with a warning limit of60%, as shown in the showvv -alert command.

# showvv —alert

-----------Alerts------------ --(MB)-- -Snp(%VSize)- -Usr(%VSize)- Adm ----Snp----- ---- Usr-----Id Name Prov Type VSize Used Wrn Lim Used Wrn Lim Fail Fail Wrn Lim Fail Wrn Lim612 nospace1 tyvv base 102400 0.0 -- -- 61.1 60 - - - - - - - y


When the warning limit is reached, the 3PAR StoreServ Storage sends a soft threshold errorasc/q: 0x38/0x7 and ESX continues to write.

InServ debug log:1 Debug Host error undefined Port 1:5:2 -- SCSI status 0x02 (Checkcondition) Host:sqa-dl380g5-14-esx5 (WWN 2101001B32A4BA98) LUN:22 LUNWWN:50002ac00264011c VV:0 CDB:280000AB082000000800 (Read10) Skey:0x06 (Unit attention)asc/q:0x38/07 (Thin provisioning soft threshold reached) VVstat:0x00 (TE_PASS-- Success) after 0.000s (Abort source unknown) toterr:74882, lunerr:2

# showalert

Id: 193State: NewMessage Code: 0x0270001Time: 2011-07-13 16:12:15 PDTSeverity: InformationalType: TP VV allocation size warningMessage: Thin provisioned VV nospace1 has reached allocationwarning of 60G (60% of 100G)

When the 3PAR StoreServ Storage runs out of disk space, a hard permanent error asc/q: 0x27/0x7is sent. Use showspace, showvv -r, and showalert to see the warning and space usage.The ESX responds by stunning the VM.

InServ debug log:1 Debug Host error undefined Port 1:5:2 -- SCSI status 0x02 (Checkcondition) Host:sqa-dl380g5-14-esx5 (WWN 2101001B32A4BA98) LUN:22 LUNWWN:50002ac00264011c VV:612 CDB:2A00005D6CC800040000 (Write10) Skey:0x07 (Data protect)asc/q:0x27/07 (Space allocation failed write protect) VVstat:0x41 (VV_ADM_NO_R5-- No space left on snap data volume) after 0.302s (Abort source unknown)toterr:74886, lunerr:3

The following figure shows the VM warning displayed on the vSphere with the Retry and Canceloptions.

Figure 3 VM Message — Retry and Cancel options


Additional New Primitives Beginning with ESXi 5.xWith 3PAROS 3.1.1 or later, additional new primitives are supported, called TP LUN Reporting,where the ESXi 5.x is notified that a given LUN is a thin provisioning LUN by enabling the TPEbits of the READ Capacity (16) and enables the host to use features such as sending the UNMAPcommand to these LUNs. The TPE bit is enabled for TPVVs and R/W snapshot of TPVV base.The Quota Exceeded Behavior present in ESXi 5.x is accomplished through the "Thin ProvisioningSoft Threshold Reached" check condition, providing alerts and warnings.

VAAI and New Feature Support TableThe following table summarizes the VAAI plug-in requirements and New Primitives support.

Table 2 3PAR VAAI Plug-in Installation Requirements

ESXi 5.x, ESXi 6.0ESX 4.1VMware ESXFeatures/Primitives

Supported. Does not require 3PAR VAAIplug-in (supported by Standard T10 ESXplug-in).

Needs 3PAR VAAI 1.1.1 plug-in

ATS

XCOPY

WRITE_SAME

Not supported by ESX 4.1 or laterUNMAP

Supported. 3PAR VAAI 1.1.1 plug-inrecommended, but not required for thisspecific feature.

Out-of-space condition (alsoknown as, “VM STUN”)

Not supported by ESX 4.1 or laterQuota Exceeded Behavior

TP LUN Reporting

VAAI Plug-in VerificationWith ESXi 5.x or ESXi 6.0:

NOTE: With ESXi 5.x or ESXi 6.0, 3PAR OS 3.1.1 uses the native T10 plug-in, and shouldnot show any 3PAR plug-in.

# esxcli storage core plugin list

Plugin name Plugin class----------- ------------NMP MP


With ESX 4.1:Verify that the VAAI Plug-in is installed and enabled on devices:• Show the version of the installed VAAI plug-in:

# esxupdate --vib-view query | grep 3par

cross_3par-vaaip-inserv_410.1.1-230815 installed

• Show that the claim rule is in effect for the 3PAR devices discovered:

# esxcli corestorage claimrule list -c VAAI

Rule Class Rule Class Type Plugin MatchesVAAI 5001 runtime vendor 3PAR_vaaip_InServ vendor=3PARdata model=*VAAI 5001 file vendor 3PAR_vaaip_InServ vendor=3PARdata model=*

• Show that the VAAI is supported on the device:

# esxcfg-scsidevs -l

naa.50002ac003da00ebDevice Type: Direct-AccessSize: 512000 MBDisplay Name: 3PARdata iSCSI Disk (naa.50002ac003da00eb)Multipath Plugin: NMPConsole Device: /dev/sdxDevfs Path: /vmfs/devices/disks/naa.50002ac003da00ebVendor: 3PARdata Model: VV Revis: 3110SCSI Level: 5 Is Pseudo: false Status: onIs RDM Capable: true Is Removable: falseIs Local: falseOther Names:

vml.020001000050002ac003da00eb565620202020VAAI Status: supported

To take advantage of the storage primitives on ESX 4.1, ESXi 5.x, or ESXi 6.0, the followingoutput shows that Hardware Acceleration is enabled on 3PAR LUNs.

Figure 4 Hardware Acceleration for 3PAR devices

Use the esxcfg-advcfg command to check that the options are set to 1 (enabled):

# esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove# esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit# esxcfg-advcfg -g /VMFS3/HardwareAcceleratedLocking


6 Configuring the Host as an FCoE Initiator Connecting toan FC target or an FCoE Target

All contents of the FC topics of this guide also apply to FCoE connectivity. See the followingtopics:

• “Install Virtual Machine Guest OS” (page 36)

• “Multipath Failover Considerations and I/O Load Balancing” (page 38)

• “Performance Considerations for Multiple Host Configurations” (page 49)

• “ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature Considerations” (page 51)This topic describes the procedures for setting up an ESX software FCoE configuration with a3PAR StoreServ Storage. These instructions cover both end-to-end FCoE and FCoE initiator toFC target. The instructions in this topic should be used in conjunction with the VMware vSphereStorage guide available at the VMware website.


Configuring the FCoE SwitchConnect the ESX (FCoE Initiator) host ports and 3PAR StoreServ Storage server (FCoE target)ports to an FCoE-enabled switch.

NOTE: FCoE switch VLANs and routing setup and configuration is beyond the scope of thisdocument. For instructions on setting up VLANs and routing, refer to the manufacturer's guidefor the switch.

Using system BIOS to configure FCoE1. Enter the setup menu. The combination of keys to press to enter setup might be different

depending on the host being configured. The example below is for an HPE ProLiant:

Figure 5 Configuring FCoE

58 Configuring the Host as an FCoE Initiator Connecting to an FC target or an FCoE Target


2. In the System Options pane, select NIC Personality Options.

Figure 6 NIC Personality Options

3. In the PCI Slot 2 Pane, select FCoE for both Port 1 and Port 2.

Figure 7 Configuring the PCI Slots

Using system BIOS to configure FCoE 59

4. PCI Slot 2 Port 1 and Port 2 now display FCoE.

Figure 8 PCI Slot 1 and Slot 2 Configured for FCoE

5. Save the changes and exit the BIOS.

Figure 9 Exiting the BIOS Utility


Configure a 3PAR StoreServ Storage Port for an FCoE Host ConnectionSetting up an FCoE initiator to FC target, does not require any special configuration on the 3PARStoreServ Storage. The initiator coming from the host adapters through the FCoE Forwarderswitch is treated as another FC device connecting to the 3PAR StoreServ Storage ports. Thesame guidelines described in “Configuring the 3PAR StoreServ Storage for FC” (page 9) and“Configure the Host for an FC Connection” (page 35) must be followed when a server with a hostCNA card configured with FCoE is connected to 3PAR StoreServ Storage ports.When setting up FCoE initiator to FCoE target, the StoreServ ports must be configured for FCoE.See “Configuring the 3PAR StoreServ Storage for FCoE” (page 28), for notes on how to configureFCoE ports on the StoreServ.For specific configurations that support FCoE CNAs and forwarder switches, refer to theappropriate 3PAR OS release version at the SPOCK website.


Configuring Initiator FCoE to FC TargetIf an FCoE to FC configuration is being set up, the figure below summarizes the general procedureto configure a CNA and FCoE Forwarder Switch.

Figure 10 Initiator FCoE to FC Target

NOTE: For complete and detailed instructions for configuring a server with a given ConvergedNetwork Adapter, see the CNA manufacturer documentation.The FCoE switch or FCoE forwarder must be able to convert FCoE traffic to FC and to trunk thistraffic to the fabric where the 3PAR StoreServ Storage target ports are connected.

Configure a 3PAR StoreServ Storage Port for an FCoE Host Connection 61


1. Install the CNA card in the server like any other PCIe card - see the server vendordocumentation for specific instructions.

2. Install the CNA card driver according to the CNA card installation instructions (it assumesthe server is already running a supported OS).

3. Physically connect the server CNA card ports to the FCoE Forwarder switch and thenconfigure the FCoE Forwarder switch ports - see the switch vendor documentation for specificinstructions.

4. Configure the 3PAR StoreServ Storage ports according to the guidelines in topic “Configuringthe 3PAR StoreServ Storage Port Running 3PAR OS 3.2.x or 3.1.x ” (page 9) and thenconnect the 3PAR StoreServ Storage port either to the FCoE Forwarder FC switch ports orthe FC fabric connected to the FCoE Forwarder.

5. Create FC zones for the host initiator’s ports and the 3PAR StoreServ Storage target port.When the initiators are logged in to the 3PAR StoreServ Storage target ports, create a hostdefinition and provision storage to the host.

NOTE: It is not possible to connect a server with a CNA directly to the 3PAR StoreServStorage. An FCoE Forwarder switch must be used.


Configuring Initiator FCoE to FCoE TargetThe following figure summarizes the general procedure for an FCoE to FCoE configuration. Whensetting up FCoE initiator to FCoE target, the 3PAR StoreServ Storage ports must be configuredfor FCoE. See “Configuring the 3PAR StoreServ Storage for FCoE” (page 28), for notes on howto configure FCoE ports on the StoreServ.

Figure 11 Initiator FCoE to Target FCoE

NOTE: FCoE switch VLANs and routing setup and configuration are beyond the scope of thisdocument. For instructions on setting up VLANs and routing, refer to the manufacturer's guidefor the switch.

1. Install the CNA card in the server like any other PCIe card - see the server vendordocumentation for specific instructions.

2. Install the CNA card driver following the CNA card installation instructions (it assumes theserver is already running a supported OS).

3. Physically connect the server CNA card ports to the FCoE fabric.4. Configure the 3PAR StoreServ Storage ports according to the guidelines in “Configuring the

3PAR StoreServ Storage for FCoE” (page 28) and connect the 3PAR StoreServ Storageports to the FCoE fabric.

5. Create VLANs for the host initiator’s ports and the 3PAR StoreServ Storage target port.Once the initiators have logged in to the 3PAR StoreServ Storage target ports, create a hostdefinition and provision storage to the host.

Configuring Initiator FCoE to FCoE Target 63

7 Configure the Host for an iSCSI ConnectionThis topic describes the procedures for setting up an ESX iSCSI configuration with a 3PARStoreServ Storage. Use the instructions in this topic in conjunction with the VMware vSphereStorage guide available at the VMware website.The contents of the following FC topics listed below also apply to iSCSI connectivity:

• “Install Virtual Machine Guest OS” (page 36)

• “Multipath Failover Considerations and I/O Load Balancing” (page 38)

• “Performance Considerations for Multiple Host Configurations” (page 49)

• “ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature Considerations” (page 51)


Enterprise iSCSIBeginning with 3PAR OS 3.2.2, the 3PAR StoreServ 20000 and 8000 Storage systems supportEnterprise iSCSI or iSCSI over data center bridging (DCB) (lossless Ethernet). DCB enhancesEthernet LANs for clustering and storage network use.With DCB, you can set up and configure different Ethernet traffic with different priorities, treatingthem as if they were different pipes. You can allocate bandwidth on links and eliminate packetloss due to queue overflow.No special settings are required on the 3PAR iSCSI target port to support Enterprise iSCSI.Since each SAN environment is different and has different needs, no single switch networkconfiguration applies to all SAN environments. To configure iSCSI on an Ethernet switch thatsupports DCB, see the switch vendor’s configuration guide.To configure the HPE 5900 Network switch series, see the HPE FlexFabric 59xx/57xx SwitchSeries at the Hewlett Packard Enterprise Network Information Library.For information about supported DCB Ethernet switches, see the SPOCK website.

More informationhttp://www.hpe.com/networking/libraryhttp://www.hpe.com/storage/spock

64 Configure the Host for an iSCSI Connection


http://www.hpe.com/networking/library


Setting Up the Switch and iSCSI InitiatorSetup and configure the host Network Interface Card (NIC) or converged network adapter (CNA)as an Initiator port that will be used by the iSCSI Initiator to connect to the 3PAR StoreServStorage iSCSI target ports.

• Connect the ESX host iSCSI initiator ports and the 3PAR StoreServ Storage iSCSI targetports to the switches.

• Configure the iSCSI initiator ports and 3PAR StoreServ Storage iSCSI target ports.

• When using VLANs, verify that the target ports and initiator ports are in the same VLANs.

• Verify the connectivity between the iSCSI initiators and the 3PAR StoreServ Storage iSCSItargets by using the vmkping command on the ESX host.

NOTE:• Setting up the switch for VLAN and routing configuration is beyond the scope of this

document. For instructions about setting up VLANs and routing, see the manufacturer'sguide for the switch.

• For instructions when configuring the jumbo frame setting on the switch, see themanufacturer's guide for the switch.

Installing Software iSCSI on VMware ESXSoftware iSCSI drivers for VMware supported NICs are included as part of the ESX OS installationpackage supplied by VMware. Get updates and patches for the Software iSCSI drivers fromVMware support.VMware Software iSCSI installation requires the following basic steps:1. Configure VMkernel ports to be used for Software iSCSI2. Add the VMware Software iSCSI adapter as a Storage Adapter3. Open the VMware firewall port (3260) to be used by the Software iSCSI client4. Enable the Software iSCSI adapter5. Add the VMkernel port IPs to be used for iSCSIFigure 12 (page 66) shows an example of an ESX 4.x Software iSCSI initiator configuration withtwo servers.

Setting Up the Switch and iSCSI Initiator 65

Figure 12 ESX 4.x iSCSI Software Initiator Configuration

NOTE:• When multiple teamed NICs are configured, all 3PAR StoreServ Storage iSCSI ports and

ESX iSCSI NICs must be in the same VLAN of the IP switch.• Hewlett Packard Enterprise recommends an Active/Active NIC Teaming configuration for

best failover performance with ESX 4.x.• With ESXi 5.x and 6.0, NIC teaming for iSCSI ports is not supported by VMware.

• With ESXi 5.x and 6.0, each iSCSI port should be configured as a separate VMkernel portand IP address.


Figure 13 (page 67) shows an example of an ESXi 5.x/6.0 iSCSI software initiator configuration.

Figure 13 ESXi 5.x/6.0 iSCSI Software Initiator Configuration

Installing Software iSCSI on VMware ESX 67

Creating a VMkernel Port for Software iSCSI (ESX 4.0 Example)1. Log into the vSphere client and select the server from the inventory panel. The hardware

configuration page for this server appears.2. Click the Configuration tab and click Networking.3. Click the Add Networking Link. The Add Network wizard appears as shown below:

Figure 14 Add Network Wizard

4. Select VMkernel and click Next to allow for connecting to the VMkernel, which runs servicesfor iSCSI storage to the physical network. The Network Access page appears.

5. Select Create a Virtual Switch and select the NICs that will be used for iSCSI. (In thisexample, 2 NICs are selected to configure an active/active teamed NICs that will connectto the 3PAR StoreServ Storage array.)


6. Click Next.7. With ESX 4.x—Configure Active/Active NIC Teaming by bringing up all of the NIC adapters

being used as Active Adapters in the vSwitch Properties.For each ESX host—Use the vSphere clientConfiguration tab→Networking→Properties,click the Edit radio button, and then highlight and use the Move Up radio button to bringeach of the NIC adapters for NIC teaming from the Standby Adapters orUnused Adapterspane to the Active Adapters pane.The dialog box shows that this was completed for NIC adapters vmnic1 and vmnic2.

Figure 15 vSwitch1 Properties

Creating a VMkernel Port for Software iSCSI (ESX 4.0 Example) 69

8. Click OK to complete.The VMkernel – Network Access pane appears.

Figure 16 VMkernel — Network Access

9. Click Next, enter a network label, and then click Next.10. Specify the IP settings for the iSCSI network and then click Next.11. Review the information and then click Finish.For detailed instructions, refer to the VMware vSphere Installation and Setup guide at the VMwarewebsite.




Configuring a Service Console Connection for the iSCSI Storage on ESX4.x

1. From the Configuration tab, and the Networking tab, click Properties for the vSwitchassociated with the VMkernel port that was just created for the iSCSI network. The vSwitch1Properties dialog box appears as shown:

Figure 17 iSCSI Network vSwitch1 Properties

2. Click Add.

Configuring a Service Console Connection for the iSCSI Storage on ESX 4.x 71

3. Select the radio button for VMkernel to add support for host management traffic:

Figure 18 Service Console – Network Access

4. Click Next.5. Enter the network label and IP address for the service console used to communicate with

the iSCSI software initiator. The IP address must be in the same subnet as the iSCSI.


6. Click Next. A window appears showing the changes or additions that were made:

Figure 19 Ready to Complete

7. Click Finish.8. Close all dialog boxes associated with the network configuration.9. Check the Configuration display:

Figure 20 VMware ESX Server—Configuration Tab

The new network configuration is displayed with the addition of the iSCSI network.Ping the 3PAR StoreServ Storage ports that were previously defined from the COS.

Configuring a Service Console Connection for the iSCSI Storage on ESX 4.x 73

Create a VMkernel Port for Software iSCSIIn order to create the iSCSI network, first create one or more VMware Networking vSwitch /VMkernel port interfaces and assigned physical NIC port combinations.

Creating a VMkernel Port in the vSphere Client (ESXi 5.1 Example)

NOTE: A Service Console Port is not part of vSwitch for iSCSI with ESXi.

1. Click the Configuration tab, and then select Networking from the Hardware options.2. Select Add Networking..., select VMkernel, and then click Next3. Under Create a vSphere standard switch, select a vmnic port that will be configured and

connected to the storage array, and then click Next.4. Assign aNetwork Label (meaningful or descriptive name for the port group), assign a VLAN

ID (VLAN tag) if desired, and then click Next.

5. Click Next to set the IP address and subnet mask of the VMkernel (vmk).


6. Click Next, and then Finish, to return to the Networking on the Configuration tab.

7. Repeat step 1 to step 6 to create a vmk for the second port. The final configuration shouldhave two vmk (vmk1 and vmk2) and vSwitches that each are on different subnets(recommended) For example: iSCSI-1 vmk2 = 10.1.1.10 and iSCSI-2 vmk3 =10.1.2.20.].

Create a VMkernel Port for Software iSCSI 75

Configuring the VMware Software iSCSI Initiator• Only one VMware Software iSCSI adapter can be added and enabled, though multiple ports

can be configured for the adapter.• Use vSphere vCenter to configure for Software iSCSI (recommended).

Adding the VMware Software iSCSI Adapter (ESXi 5.1 Example)The following is an example of a Software iSCSI adapter configuration on an ESXi 5.1 host usingthe vSphere Client to gain access to vCenter.1. Log in to the vSphere Client, and then select a host from the inventory panel.2. Click the Configuration tab, and then click Storage Adapters from the Hardware options.3. Select Add Software iSCSI Adapter from the Add Storage Adapter pop-up menu, and

then click OK.

4. ClickOK through messages and a note stating that a vmhba adapter named “iSCSI SoftwareAdapter” has been added to the list of adapters on theConfiguration→Hardware→StorageAdapters page.


5. With the Configuration tab selected, select Security Profile from the Software options,and then select Firewall Properties as shown:

6. Open the firewall ports that will be used for the iSCSI connection, and then click OK.7. The iSCSI Software Initiator must be enabled before the ESX host can use it. Click the

Storage Adapters option from the Hardware options:

Configuring the VMware Software iSCSI Initiator 77

8. In the ESX host, select the Configuration tab. Then select iSCSI Software Adapter inStorage Adapter.

9. Click the Properties tab.10. Select the General tab.11. Click Configure...12. Select the Enabled check box for the status.13. Click OK.14. Click the Dynamic Discovery tab. Dynamic discovery enables the Send Targets discovery

method, where the initiator sends the request to discover and log into the targets:

15. Click Add....16. Enter the IP address of one of the previously defined 3PAR StoreServ Storage iSCSI ports.17. Click OK.


18. Add additional 3PAR StoreServ Storage iSCSI ports if they exist and have been defined onthe system:

19. When all of the 3PAR StoreServ Storage iSCSI ports have been added to the DynamicDiscovery tab, close this window.

Configuring the VMware Software iSCSI Initiator 79

20. A host reboot is required to complete configuration.The ESX host and 3PAR StoreServ Storage should now be configured for use. When usingthe showhost command on the 3PAR StoreServ Storage, the new iSCSI connections shouldnow show as present.

# showhost

Id Name ----------------WWN/iSCSI_Name---------------- Port-- iqn.1998-01.com.vmware:hpdl380-01-11a38a59 0:1:2

iqn.1998-01.com.vmware:hpdl380-01-11a38a59 1:1:2

As new LUNs are exported to the ESX iSCSI host, a re-scan must be performed on theiSCSI software adapter.

ClickRescan, select the Scan for New Storage Devices check box, then clickOK to re-scanfor new LUNs exported to the ESX iSCSI host.


Setting Up and Configuring Challenge-Handshake Authentication ProtocolEnabling Host Challenge-Handshake Authentication Protocol (CHAP) is an option set at the ESXsystem administrator's discretion. Both host CHAP (initiator) and mutual CHAP (bidirectional,initiator-target) are supported. The following example outlines the procedures for host (initiator)CHAP enablement with Software iSCSI.See “Independent Hardware iSCSI” (page 83) for CHAP enablement with Independent HardwareiSCSI.1. Use the 3PAR CLI showhost command to verify that a host definition was created on 3PAR

StoreServ Storage for the ESX host that will have CHAP enabled.

# showhost

Id Name ----------------WWN/iSCSI_Name---------------- Port0 ESX1 iqn.1998-01.com.vmware:hpdl380-01-11a38a59 0:1:2

iqn.1998-01.com.vmware:hpdl380-01-11a38a59 1:1:2

2. On the 3PAR StoreServ Storage, use the 3PAR CLI sethost command with the initchapparameter to set the CHAP secret for the ESX host.The following example uses the CHAP secret (CHAP password) host_secret3 for theESX host. The CHAP secret must be at least 12 characters long.

# sethost initchap -f host_secret3 ESX1

NOTE: If mutual CHAP on ESX is configured, configure the target CHAP on the 3PARStoreServ Storage as well as the initiator CHAP. Set the target CHAP secret by using the3PAR CLI sethost command and the targetchap parameter.

# sethost targetchap -f secret3_host ESX1

NOTE: The initiator CHAP secret and the target CHAP secret cannot be the same.

3. Use the 3PAR CLI showhost -chap command to verify that the specified CHAP secretwas set for the host definition.For Initiator chap:

# showhost -chap

Id Name -Initiator_CHAP_Name- -Target_CHAP_Name-0 ESX1 ESX1 --

For Mutual chap:

# showhost -chap

Id Name -Initiator_CHAP_Name- -Target_CHAP_Name-0 ESX1 ESX1 s331

Setting Up and Configuring Challenge-Handshake Authentication Protocol 81

Configuring CHAP on ESX/ESXi Host1. On the ESX host vSphere client, select the Configuration tab, and then click Storage

Adapters under the Hardware pane. Select the iSCSI Software Adapter, select theProperties link and click CHAP.Click to select Use initiator name to set the CHAP name to the iSCSI adapter name.

Figure 21 CHAP Credentials

2. Enter the Mutual CHAP Secret.Use the same CHAP secret on both the ESX host and the 3PAR StoreServ Storage.

3. When configuring mutual CHAP on ESX, specify the incoming CHAP credentials. Use the3PAR StoreServ Storage name as the Mutual CHAP Name.Use the 3PAR CLI showsys command to get the Mutual CHAP Name (name of the 3PARStoreServ Storage).

# showsys

---------------(MB)----------------ID -Name- ---Model---- -Serial- Nodes Master TotalCap AllocCap FreeCap FailedCap99800 s331 HP_3PAR 7200 1699800 2 1 8355840 849920 7504896 1024

4. Click OK, and then close the Properties window. A warning appears indicating that eithera restart of the ESX host or a storage adapter re-scan is required.To perform this stepmanually, restart the host or remove and then add the iSCSI sessionsby using either the esxcli iscsi session command or the vicfg-iscsi command,depending on the ESX version.


Hardware iSCSI SupportConfigure supported CNA in either of the following modes:Dependent iSCSI Obtain the IP address from the host NIC connections.Independent iSCSI Enter the IP address into the CNA card BIOS.Independent iSCSI configurations can be configured to boot from SAN; iSCSI targets are enteredinto the card via CNA BIOS.For more information about supported models and configurations, see the SPOCK website.

More informationhttp://www.hpe.com/support/hpeschttp://www.hpe.com/storage/spock

Independent Hardware iSCSIThe following example uses the CN1100E CNA:1. After installing the CN1100E, boot the ESX host. On boot up, do the following:

Use <Ctrl> <P> to change the CNA personality from FCoE to iSCSI ifnecessary.

2. To set a static IP address: Press Ctrl+S to enter the utility.

Emulex 10Gb iSCSI Initiator BIOS..Press <Ctrl> <S> for iSCSISelect(TM) Utility

Figure 22 iSCSI Utility

3. Select a controller and then press Enter.4. On the Controller Configuration screen, select Network Configuration and then press

Enter.

Hardware iSCSI Support 83



5. On theNetwork Configuration screen, selectConfigure Static IP Address and then pressEnter. The screen for setting a static IP address appears.

Figure 23 Setting a Static IP Address

6. Type the IP address, subnet mask, default gateway and then click Save to return to theController Configuration menu.

Adding an iSCSI Target1. If this is a boot from SAN configuration, select Controller Properties from the Controller

Configuration menu.On the properties screen, verify that boot support is enabled. If it is not, scroll toBoot Supportand enable it, then save and exit this screen.

2. On the Controller Configuration menu, select iSCSI Target Configuration.3. Select Add New iSCSI Target and then press Enter.4. Fill in the information for the first iSCSI target. Make sure Boot Target is set to Yes.

Figure 24 Adding an iSCSI Target

5. Click Ping to verify connectivity.6. After a successful ping, click Save/Login.


7. When both controllers are configured, use the showiscsisession command to displaythe iSCSI sessions on the 3PAR StoreServ Storage. If everything is configured correctly,the output should be similar to the following example:

root@jnodec103140:S99814# showiscsisession

0:2:1 10.101.0.100 21 15 1 iqn.1990-07.com.emulex:a0-b3-cc-1c-94-e1 2012-09-24 09:57:58 PDT1:2:1 10.101.1.100 121 15 1 iqn.1990-07.com.emulex:a0-b3-cc-1c-94-e1 2012-09-24 09:57:58 PDT

root@jnodec103140:S99814# showhost -d Esx50Sys1

1 Esx50Sys1 VMware iqn.1990-07.com.emulex:a0-b3-cc-1c-94-e1 0:2:1 10.101.0.1001 Esx509Sys1 VMware iqn.1990-07.com.emulex:a0-b3-cc-1c-94-e1 1:2:1 10.101.1.100

8. Configuring CHAP as an Authentication Method. From the Add/Ping iSCSI Target screen,select Authentication Method, and choose One-Way CHAP from the list as shown below:

Figure 25 One-Way CHAP

When the CHAP Configuration screen appears, type the Target CHAP name (the initiatorIQN name) and Target secret as shown below:

Figure 26 CHAP Configuration for One-Way CHAP

Hardware iSCSI Support 85

In the Authentication Method setting on the Add-Ping iSCSI Target screen select MutualCHAP. The CHAP Configuration screen appears as shown:

Figure 27 CHAP Configuration for Mutual CHAP

Fill in the Target CHAP Name (the initiator IQN name), the Target Secret, the InitiatorCHAP Name (DNS name of the storage), and an Initiator Secret, click OK.To remove CHAP authentication, in the Authentication Method setting on the Add-PingiSCSI Target screen, select None.

9. When setting up CHAP authentication before rebooting the host system, make sure to setthe matching CHAP parameters for the host in the 3PAR StoreServ Storage.• If one-way CHAP was selected, enter the matching CHAP secret as follows:

root@jnodec103140:S99814# sethost initchap -f aaaaaabbbbbb EsxHost1root@jnodec103140:S99814# showhost -chap

• If mutual CHAP was selected, enter the mutual CHAP secret as follows:

root@jnodec103140:S99814# sethost initchap -f aaaaaabbbbbb EsxHost1root@jnodec103140:S99814# sethost targetchap -f bbbbbbcccccc EsxHost1root@jnodec103140:S99814#root@jnodec103140:S99814# showhost -chap

Id Name -Initiator_CHAP_Name- -Target_CHAP_Name-1 EsxHost1 EsxHost1 S814root@jnodec103140:S99814#

10. After entering the CHAP secret, exit the BIOS and reboot the host.


Installing Virtual Machine Guest Operating SystemThe VMware ESX documentation lists recommended VM guest OS and their installation andsetup as VMs. For information about setting up the VM configuration, refer to the VMware ESXdocumentation at the VMware website.

CAUTION: In VMware KB 51306, VMware identifies a problem with RHEL 5 (GA), RHEL 4 U4,RHEL 4 U3, SLES 10 (GA), and SLES 9 SP3 guest operating systems. Their file systems mightbecome read-only in the event of busy I/O retry or path failover of the ESX host’s SAN or iSCSIstorage. Refer to KB 51306 at the VMware Knowledge Base website.Hewlett Packard Enterprise does not recommend and does not support the use of RHEL 5 (GA),RHEL 4 U4, RHEL 4 U3, SLES 10 (GA), and SLES 9 SP3 as guest operating systems for VMson VMware ESX hosts attached to 3PAR StoreServ Storage systems.

NOTE:• VMware and Hewlett Packard Enterprise recommend the LSI logic adapter emulation for

Windows 2003 Servers. The LSI Logic adapter is also the default option for Windows 2003when creating a new VM. Hewlett Packard Enterprise testing has noted a high incidence ofWindows 2003 VM failures during an ESX multipath failover/failback event when the BUSLogic adapter is used with Windows 2003 VMs.

• Hewlett Packard Enterprise testing indicates that the SCSI timeout value for VM guest OSsshould be 60 seconds to successfully ride out path failovers at the ESX layer. Most guestoperating systems supported by VMware have a default SCSI timeout value of 60 seconds,but this value should be checked and verified for each guest OS installation. Change theSCSI timeout value in Red Hat 4.x guest operating systems from their default value of 30seconds to 60 seconds.This command can be used to set the SCSI timeout on all SCSI devices presented to a RedHat 4.x VM to 60 seconds:find /sys -name timeout | grep "host.*target.*timeout" | xargs -n1 echo "echo 60 >"|sh

Add this line in the /etc/rc.local file of the Red Hat 4.x guest OS for the timeout changeto be maintained with a VM reboot.Example of a modified /etc/rc.local file:

# cat /etc/rc.local

#!/bin/sh

#

# This script will be executed *after* all the other init scripts.

# You can put your own initialization stuff in here if you don't

# want to do the full Sys V style init stuff.

find /sys -name timeout | grep "host.*target.*timeout" | xargs -n 1 echo "echo 60 >"|sh

touch /var/lock/subsys/local

More informationhttp://www.vmware.comhttp://kb.vmware.com

Installing Virtual Machine Guest Operating System 87



iSCSI Failover Considerations and Multipath Load BalancingRefer to “Multipath Failover Considerations and I/O Load Balancing” (page 38) for configurationof VMware multipath and I/O load balancing.

Performance Considerations for Multiple Host ConfigurationsRefer to the information in the “Performance Considerations for Multiple Host Configurations”(page 49) topic for multiple hosts connected in a fan-in configuration to 3PAR StoreServ Storageports.

NOTE: When running W2K8 VM Cluster with RDM-shared LUNs, then individually changethese specific RDM LUNs from Round Robin policy to Fixed Path policy.

ESX/ESXi Additional Feature ConsiderationsRefer to the “ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0 Additional Feature Considerations” (page 51)topic for considerations of additional VMware features including enabling of VAAI functions.


8 Allocating Storage for Access by the ESX HostThis topic describes the basic procedures required for creating and exporting VVs so they canbe used by the VMware ESX host. For complete details on creating and managing storage onthe 3PAR StoreServ Storage, see the appropriate 3PAR documentation.

Create Storage on the 3PAR StoreServ StorageThis topic describes the general procedures and commands that are required to create the VVsthat can then be exported for discovery by the host.For additional information, see the HPE 3PAR Command Line Interface Administrator’s Manual.For a comprehensive description of 3PAR OS commands, see the HPE 3PAR Command LineInterface Reference at the Hewlett Packard Enterprise Storage Information Library website.In addition, for a detailed discussion about using TPVV and strategies for creating VLUNs, see3PAR Utility Storage with VMware vSphere at the Hewlett Packard Enterprise Support Centerwebsite.

More informationhttp://www.hpe.com/info/storage/docshttp://www.hpe.com/support/hpesc

Create Thinly Deduplicated Virtual Volumes

NOTE: With 3PAR OS 3.2.1 MU1 and later, 3PAR Thin Deduplication feature is supported.To create TDVVs (thinly deduplicated virtual volumes), a 3PAR Thin Provisioning license isrequired.

3PAR Thin Deduplication allows the creation of TDVVs from solid state drive (SSD) CPGs. ATDVV has the same characteristics as a TPVV, with the additional capability of removingduplicated data before it is written to the volume. The TDVVs are managed like any other TPVV.A TDVV must be associated with CPGs created from an SSD.For more information about 3PAR Thin Deduplication, see the following documents:

• HPE 3PAR StoreServ Storage Concepts Guide

• HPE 3PAR Command Line Interface Administrator’s Manual

• HPE 3PAR OS Command Line Interface Reference

• HPE 3PAR Thin Technologies—Technical white paper containing best practicesThese documents are available at the Hewlett Packard Enterprise Storage Information Librarywebsite.

More informationhttp://www.hpe.com/info/storage/docs

Create Storage on the 3PAR StoreServ Storage 89




Create Virtual VolumesVVs are the only data layer visible to hosts. After devising a plan for allocating space for the ESXhost, create the VVs on the 3PAR StoreServ Storage.Create volumes that are provisioned from one or more Common Provisioning Groups (CPGs).Volumes can be either fully provisioned, thinly provisioned, or thinly deduplicated volumes.Optionally, specify a CPG for snapshot space for provisioned volumes.

Creating Virtual Volumes using the 3PAR Management Console (MC)1. From the menu bar, select:

Actions→Provisioning→Virtual Volume→Create Virtual Volume2. Use the Create Virtual Volume wizard to create a base volume.3. Select one of the following options from the list:

• Fully Provisioned• Thinly Provisioned• Thinly Deduplicated (Supported with 3PAR OS 3.2.1 MU1 and later)

Creating Virtual Volumes using the 3PAR StoreServ Management Console (SSMC)1. On the main menu, select:

Block Persona→Virtual Volume2. Click + Create virtual volume or select Create on the Actions menu.3. Follow the instructions on the dialog that opens and select one of the following virtual volume

options:• Fully Provisioned• Thinly Provisioned• Thinly Deduplicated (Supported with 3PAR OS 3.2.1 MU1 and later)

90 Allocating Storage for Access by the ESX Host

Creating Virtual Volumes using the 3PAR CLICreate a fully provisioned or TPVV:

cli % createvv [options] <usr_CPG> <VV_name> [.<index>] <size>[g|G|t|T]

For example:

cli % createvv -cnt 5 testcpg TESTLUNS 5g

For complete details on creating volumes for the 3PAR OS version that is being used on the3PAR StoreServ Storage, see the following documents:

• HPE 3PAR StoreServ Management Console User Guide

• HPE 3PAR Management Console User Guide

• HPE 3PAR OS Command Line Interface ReferenceThese documents are available at the Hewlett Packard Enterprise Storage Information Librarywebsite.

NOTE: The commands and options available for creating a VV might vary for earlier versionsof the 3PAR OS.


Create Storage on the 3PAR StoreServ Storage 91


Exporting LUNs to the HostThis topic explains how to export VVs created on the 3PAR StoreServ Storage as VLUNs for theESX host.When exporting VLUNs to the ESX host, be aware of the following:

• New VLUNs that are exported while the host is running might not be registered until a BusRe-scan is initiated. This might be performed from the vSphere client Management Interface.Some versions of ESX will automatically scan for newly exported LUNs.

• Disks can be added to a VM with the VM powered up. To remove a disk, the VM might notneed to be powered off. See the VMware documentation for feature support.

• The maximum number of LUNs on a single ESX HBA port is 256, and there are 256 totalLUNs on the ESX host. Internal devices, such as local hard drives and CD drives, are countedas a LUN in the ESX host LUN count.

• VLUNs can be created with any LUN number in the range from 0 to 255 (ESX limitation).

• iSCSI LUNs and FC LUNs are treated as any other LUN on the ESX host. No specialrequirements or procedures are needed to use iSCSI LUNs. Hewlett Packard Enterprise doesnot recommend or support the same storage LUN being exported on different protocolinterfaces, such as exporting to an FC interface on one host and an iSCSI on another host.This is because the timing and error recovery of the protocols would be different.

• The ESX 4.x limitation for the largest LUN that can be utilized by the ESX host is 2047 GB.For VMware ESX/ESXi 5.x or VMware ESX/ESXi 6.0, the maximum LUN size is 16 T or16384 GB.

• Sparse LUN numbering, that is, LUN numbers can be skipped, is supported by VMwareESX host. A LUN 0 is not required.

For failover support, VVs should be exported down multiple paths to the host simultaneously. Tofacilitate this task, create a host definition on the 3PAR StoreServ Storage that includes theWWNs of multiple HBA ports on the host and export the VLUNs to that host definition.Provisioning several VMs to a smaller number of large LUNs, versus a single VM per single LUN,provides better overall results. Further examination and explanation of this recommendation isoutlined in the HPE 3PAR Utility Storage with VMware vSphere document available at HewlettPackard Enterprise Support Center website.Concerning TPVVs, ESX VMFS-3 and ESX VMFS-5 do not write data to the entire volume atinitialization and can be used with TPVVs without any configuration changes to VMFS. A furtherexamination of this subject, recommendations, and limitations are explored in HPE 3PAR UtilityStorage with VMware vSphere.

More informationhttp://www.hpe.com/support/hpesc



Create a VLUN for ExportCreation of a VLUN template enables export of a VV as a VLUN to one or more ESX hosts.There are four types of VLUN templates:• port presents - created when only the node:slot:port are specified. The VLUN

is visible to any initiator on the specified port.• host set - created when a host set is specified. The VLUN is visible to the initiators

of any host that is a member of the set.• host sees - created when the hostname is specified. The VLUN is visible to the

initiators with any of the host’s WWNs.• matched set - created when both hostname and node:slot:port are specified.

The VLUN is visible to initiators with the host’s WWNs only on the specified port.For complete details on exporting volumes and available options for the 3PAR OS version usedon the 3PAR StoreServ Storage, refer to the HPE 3PAR Management Console Users Guide andtheHPE 3PARCommand Line Interface Reference documents at the Hewlett Packard EnterpriseStorage Information Library website.

NOTE: The commands and options available for creating a virtual volume might vary for earlierversions of the 3PAR OS.


Exporting LUNs to the Host 93


Exporting the LUNsExport the LUNseither through the 3PAR Management Console or the 3PAR CLI.Using the 3PAR Management Console:1. From the Menu bar, select Actions→Provisioning→VLUN→Export Volume.2. Use the Export Virtual Volume dialog box to create a VLUN template.Using the 3PAR CLI:Create a port presents VLUN template:

# createvlun [options] <VV_name | VV_set> <LUN> <node:slot:port>

Create a host sees or host set VLUN template:

# createvlun [options] <VV_name | VV_set> <LUN> <host_name/set>

Create a matched set VLUN template:

# createvlun [options] <VV_name | VV_set> <LUN> <node:slot:port>/<host_name>

Create a host set VLUN template:

# createvlun [options] <VV_name | VV_set> <LUN> <host_set>

For example:

# createvlun -cnt 5 TESTLUNs.0 0 hostname/hostdefinition

or:

# createvlun -cnt 5 TESTVLUN.0 0 set:hostsetdefinition


Discovering LUNs on VMware ESX HostsThis topic provides tips for discovering LUNs used by the ESX host.Once the host is built, the preferred method for configuring and managing the use of the ESXhost is through a vSphere client Management Interface and VMware vCenter Server.New VLUNs exported while the host is running will not be registered until a bus re-scan is initiated,this can be done automatically by ESX 4.x, ESXi 5.x, or ESXi 6.0 hosts managed by vSphereclient or VMware vCenter Server from the vSphere client Management Interface. If therecommended failover support is used, view all LUNs and their respective paths by using themenu from the ESX (Configuration tab →Storage Adapter).Disks can be added to a VM with the VM powered up. However, to remove a disk, the VM mustbe powered off. This is a limitation of ESX.The maximum number of LUNs on a single ESX HBA port is 256, and there are 256 total LUNson the ESX host. Internal devices, such as local hard drives and CD drives, are counted as aLUN in the ESX host LUN count.

Discovering LUNs on VMware ESX Hosts 95

Removing VolumesAfter removing a VLUN exported from the 3PAR StoreServ Storage, perform a ESX host busadapter re-scan. ESX will update the disk inventory upon re-scan. This applies to FC, FCoE, andiSCSI.

Figure 28 Re-scanning the ESX Host

Remove the disk/LUN from the VM inventory, detach from the ESX host, and then remove it fromthe 3PAR StoreServ Storage by using the removevlun and removevv 3PAR CLI commandsor by using the 3PAR Management Console. If a LUN is not detached but is removed from the3PAR StoreServ Storage, it appears as a device in an error state, and is cleared after an ESXhost reboot.

Figure 29 Detaching a LUN


Host and Storage Usage—Eventlog and Host Log MessagesTo see all 3PAR system debug messages, use the 3PAR CLI showeventlog -debug command.On the ESX host, errors are reported in /var/log/vmkernel files. Use the 3PAR CLIshowalert command to see alerts and warnings posted from the 3PAR StoreServ Storage.Examples of storage server posted messages:

• If you have set the space allocation warning for a TPVV volume, the system event log showsthe Thin provisioning soft threshold reached message when it reaches thatlimit. For example:

1 Debug Host error undefined Port 1:5:2 -- SCSI status 0x02 (Check condition) Host:sqa-dl380g5-14-esx5(WWN 2101001B32A4BA98) LUN:22 LUN WWN:50002ac00264011c VV:0 CDB:280000AB082000000800 (Read10) Skey:0x06 (Unitattention) asc/q:0x38/07 (Thin provisioning soft threshold reached) VVstat:0x00 (TE_PASS -- Success)after 0.000s (Abort source unknown) toterr:74882, lunerr:2

• If you have set a hard limit on the 3PAR StoreServ Storage volume when using 3PAR OS3.1.1 or later, the 3PAR StoreServ Storage will post the write protect error (ASC/Q:0x27/0x7) when the limit is reached. This is considered a hard error, and the VM is stunnedin ESX 4.x and ESXi 5.x. Additional space must be added to the storage to clear this errorcondition.

PDT 1 Debug Host error undefined Port 1:5:2 -- SCSI status 0x02 (Check condition)Host:sqa-dl380g5-14-esx5 (WWN 2101001B32A4BA98) LUN:22 LUN WWN:50002ac00264011c VV:612 CDB:2A00005D6CC800040000(Write10) Skey:0x07 (Data protect) asc/q:0x27/07 (Space allocation failed write protect) VVstat:0x41 (VV_ADM_NO_R5

-- No space left on snap data volume) after 0.302s (Abort source unknown) toterr:74886, lunerr:3

For ESX 4.x. ESXi 5.x, or ESXi 6.0, an ESX host managed by vSphere client or VMwarevCenter Server will scan for LUNs at 5-minute intervals by using the REPORT LUN commandand discovering any new LUNs.At an interval of every 5 minutes, an ESX host such as ESXi 5.x sends REPORT LUNcommands and discovers any new LUNs.On ESXi 5.x, the following vmkernel log message is not harmful and can be ignored:

34:41.371Z cpu0:7701)WARNING: ScsiDeviceIO: 6227: Thedevice naa.50002ac00026011b does not permit the system to change the sitpua bit to 1.

Each ESX host has its own limits such as volume size, virtual disk size and so on. It is notalways known if it will fully utilize the maximum volume size of 16TB from the storage array.

Host and Storage Usage—Eventlog and Host Log Messages 97

9 Boot the VMware ESX Host from the 3PAR StoreServStorage

This topic provides a general overview of the procedures required to boot the VMware ESXoperating system from the SAN.In a boot-from-SAN environment, each ESX host operating system is installed on the 3PARStoreServ Storage, instead of on the internal disk. In this situation, create a separate VV for eachESX host used for the boot image.General tasks in this process:

• Perform the required zoning

• Create a virtual volume and export it as a VLUN to the ESX host

• Boot the system and enter the HBA BIOS

• Enable the HBA port to boot

• Discover the LUN and designate it as bootable through the HBA BIOSFor detailed information, see the VMware FC SAN Configuration Guide.For information about setting a CN1100E CNA to boot the host from SAN, see “Hardware iSCSISupport” (page 83).The VMware ESX host has an option that allows the VMware Base OS to be installed and bootedfrom a SAN or 3PAR StoreServ Storage virtual storage device. Choose this option during theinitial installation phase of the VMware Server Installation. See the VMware documentation forfurther information regarding 'SANboot'.

Preparing the Host HBAs for a SAN Boot DeploymentHewlett Packard Enterprise makes the following general recommendations for preparing the hostHBAs for a SAN boot deployment:

NOTE: The NVRAM settings on HBAs can be changed by any server where they are installed.These settings persist in the HBA even after it is removed.

Returning All NVRAM Settings to Default Values to Have the Correct Settingsfor this Configuration1. After installation of the HBAs, reset all of the HBA NVRAM settings to their default values.

NOTE: Each host adapter port is reported as a host bus adapter and the HBA settingsshould be set to default.

2. Enter the host adapter setup program during server boot by pressing the combination ofkeys indicated by the host adapter.Change and save all HBA settings to their default values.

NOTE: When using a McDATA fabric, set the HBA topology to 'point to point’.There might be other vendor HBAs not listed here with different setup entry key combinations.

3. Reboot the host computer.

98 Boot the VMware ESX Host from the 3PAR StoreServ Storage

10 Using VMware VVols (VMware Virtual Volumes) with3PAR StoreServ Storage

VMware introduced the VVols feature in VMware vSphere 6.0. VMware VVols are the VMwareimplementation of software-defined storage. VMware VVols enable vSphere to provision arrayvolumes which VMs will use. This transfers the task of provisioning and managing storage volumesfor VMs from the storage array administrator to the VMware vSphere administrator.With traditional storage volumes, the storage administrator creates storage volumes manuallywith the necessary capabilities to meet the VMware administrator’s requirements. With VVols,when creating individual VMs, the array advertises its capabilities and allows the VMware vSpherevCenter administrator to create VVols as needed with the requested capabilities.The VVols required for a VM are created as needed on the 3PAR StoreServ Storage array,including the following:• configuration VVols (Config VVol)

• virtual disk VVols (VMDK VVols, Data VVol)

• swap VVols (Swap VVol)

• live-memory snapshot VVols (Mem VVol)—if needed

99

These terms are the most commonly used VMware VV terms.Virtual Volume A VVol is a container that encapsulates VM files, virtual

disks, and their derivatives. A single VM is made up ofseveral VVols, including one for configuration data, onefor each virtual disk that a VM has, one for VM swap, andadditional ones for the memory and data from any VMsnapshots. VVols are created automatically for any VMoperations, which include creation, powering on, cloning,and snapshotting VMs.

NOTE: 3PAR StoreServ Storage LUNs are also referredto as “virtual volumes” and are not related to VVols.

VASA Provider A VASA Provider is the software component that mediatesout-of-band communication (control path) for VVols trafficbetween vCenter Server, ESXi hosts, and a storage array.The VASA Provider passes information about storagetopology, capabilities, and status to vCenter Server andESXi hosts. A VASA Provider can reside either within astorage array or be external on a physical server or virtualmachine.

Protocol Endpoint ESXi hosts do not have direct access to VVols on a storagearray and instead must use a logical I/O proxy that isreferred to as a PE to communicate with VVols. The PEserves as the data path between ESXi hosts to VMs andtheir respective VVols. Storage arrays that support multiplestorage I/O paths and storage protocols (i.e., FC, iSCSI,NFS) can have multiple PEs that point to the same StorageContainer.

Storage Container Instead of using LUNs that are configured on the storagearray, VVols use Storage Containers, a pool of raw storagecapacity that becomes a logical grouping of VVols. AllVVols are created inside the Storage Container. StorageContainers are not visible via the in-band data path. TheVASA Provider manages Storage Containers and reportstheir existence to the vCenter Server and ESXi hosts viathe out-of-band control path.

Storage Profile (Policy) The storage profile is a set of rules that define quality ofservice (QoS) requirements for VMs based on the differentcapabilities provided by the physical storage array. Thesecan be used to perform placement decisions, admissioncontrol, QoS compliance monitoring, and dynamic resourceallocation management of storage resources.

Storage Capability The VASA Provider provides capability information fromthe storage array to vSphere in the form of specificattributes about physical storage resources. This caninclude capabilities of 3PAR StoreServ Storage arrays suchas RAID level, Thin Provisioning, Drive Type, Zero Detect,Snapshots, and much more. These capabilities can beunique to a specific system and are then used in storagepolicies to define levels of service that can be used to setperformance, capacity, and availability requirements forVMs.

100 Using VMware VVols (VMware Virtual Volumes) with 3PAR StoreServ Storage

Verify Licenses for VMware VVol SupportRequired License for VMware VVol Support:

• Virtual Copy license—RequiredRecommended License for VVol:

• Thin Provisioning license—Not requiredRecommended Licenses to Enhance the VVol Experience:

• Adaptive Flash Cache—Not required

• Thin Persistence—Not required

• Thin Deduplication—Not required

• Virtual Domain—Not requiredBeginning with 3PAR OS 3.2.1, the 3PAR VMware VVol support was initially introduced.Beginning with 3PAR OS 3.2.2, additional features were added and the setup for VVol hasminor differences.For specific interoperability support information on VVol with 3PAR StoreServ Storage, refer tothe SPOCK website.


Verifying Installed LicensesTo verify installed licenses, use the showlicense command.

cli% showlicense

License key was generated on Wed Oct 2 10:56:38 2013

License features currently enabled:DomainsInForm SuiteThin PersistenceThin Provisioning (102400G)Virtual CopyVirtual Copy (102400G)

Verify Licenses for VMware VVol Support 101


VASA ProviderThe VASA 2.x Provider Service is included as a component of the 3PAR OS and no additionalsoftware must be installed to use the service with 3PAR OS 3.2.1 or later.

NOTE: 3PAR StoreServ Storage array operations that include a reboot/reset of the arraymaster node, including a 3PAR OS upgrade, might disrupt VM operations initiated by vSphere.For this reason, Hewlett Packard Enterprise recommends not performing VM operations on VMsusing VVols during an array master node reboot/reset or 3PAR OS upgrade.VM operations that might fail during an array master node reboot/reset or upgrade include:• Creating a VM in a VVol datastore

• Powering up or shutting down a VM that is in a VVol datastore, or has VVol disks

• Adding or removing a VVol to/from a VM

• Snapshotting a VM or promoting a VM snapshot on a VM that has VVol disks

• Migrating a VM to/from a VVol datastore

• Cloning a VM from/to a VVol datastore

• Browsing a VVol datastore

• Attempting to register a VM in a VVol datastoreVVol-based VMs that are currently running will continue to operate normally.


VASA Provider Protocol Endpoints (PE) and 3PARIn the VMware VVol schema, the Protocol Endpoint (PE) provides the in-band pathway for accessto the VVol objects. From a SCSI protocol implementation perspective, the PE has the samecharacteristics as a traditional volume, but it is identified as a protocol endpoint (no storage spaceassociated with it) instead of a traditional volume on both the ESX host and the vCenter. Withthe 3PAR OS, there is no separate management interface to create protocol endpoints. Instead,a single PE is established automatically for a given 3PAR array when an existing vSphere (ESXi)host that is connected to the array is configured with host VMware Persona 11 or a new host iscreated on the array with host VMware Persona 11.When presented, the PE appears as a disk on the ESXi host with LUN number 256 and a VMwarenaa number that reflects the array node WWN. The ESXi hosts acquire the PE LUN during LUNdiscovery at host boot.Verify discovery of the PE LUN by using either the esxcli storage core device list--pe-only command or the esxcli storage vvol protocolendpoint list commandon the ESXi host.

Figure 30 Example of a PE LUN identified on a host

VASA Provider Protocol Endpoints (PE) and 3PAR 103

Also, the PE LUN (LUN 256) appears in the Storage Devices list in vCenter. Multipath attributesare applied to the PE LUN, whether set by SATP/PSP rule or applied manually. The VVolssubsequently have the Multipath attributes of the PE LUN.

Figure 31 VMware vCenter Storage Devices list

The PE LUN appears in the vCenter as a standard storage device.

NOTE: The VMware VVol feature including the PE LUN requires ESXi 6.0 host HBA/CNAdrivers that support VVols. When using host HBA/CNA drivers that do not support VVols, the PELUN might not be properly acquired and identified, and VMware generated warning messagesrelated to the PE LUN might appear in the ESXi 6.0 vmkernel logs. However, the error does notinterfere with non-VVol operation. For supported ESXi 6.0 drivers that support VVols, see theSPOCK website.




VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.1 SetupSetting up VVols on the 3PAR StoreServ Storage with 3PAR OS 3.2.1

1. “Understanding Certificates” (page 105)2. “Enabling the 3PAR VASA Provider” (page 106)3. “Assigning 3PAR Administrative User for VVols and Setting Up Storage Container” (page

107)4. “Allocate Common Provisioning Groups (CPG) for VVols on the array” (page 108)5. “Registering the 3PAR VASA Provider with VMware vCenter” (page 111)

Understanding CertificatesCreation and management of VMware VVols on the 3PAR array occurs through communicationbetween the VMware vCenter and the 3PAR VASA Provider on the array. This requires IP networkconnectivity between the VMware vCenter and the 3PAR array management network.To provide secure communication, vSphere components use SSL to communicate with the VASAProvider on the 3PAR array. The vSphere environment and the VASA Provider (at the requestof the vSphere environment) dynamically generate these SSL certificates. In OS 3.2.1, onlyvSphere-Managed certificate is supported. Thus, no certification management is required fromthe users. In 3PAR OS 3.2.1, only vSphere Managed certificate is supported.Before deploying VVols, be sure all ESXi hosts, vCenter Server, and storage array are in sync.If synchronization requires making adjustments to the clocks on the ESXi host and vCenter Serverafter they were set up, the vSphere Managed Certificate (VMCA) assigned to them might beincorrect and might need to be recreated.

CAUTION: Hewlett Packard Enterprise recommends configuring all systems to use a commonNTP server for date and time synchronization. Failure to do so can cause failures either whileregistering the VASA Provider with VMware vCenter, attempting to mount a VVol StorageContainer as a Datastore, or when creating a VM.After adjusting the date on the array, be sure to regenerate the SSL cerificate for the VASAProvider by using the setvasa -reset command.

IMPORTANT: Only use the setvasa -reset command when the VASA Provider has notbeen registered with the vSphere environment.

VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.1 Setup 105

Enabling the 3PAR VASA ProviderTo use the vSphere environment to manage VVols, enable the VASA 2.0 protocol on the storagearray, and then establish an IP connection for communication between the VMware vCenter andthe array. The 3PAR VASA Provider enables communication between the vCenter and the 3PARStoreServ Storage array management interface by using the VMware VASA 2.0 Protocol.To enable the VASA 2.0 protocol, start the VASA Provider service can be started and enabledon the storage array using the 3PAR CLI startvasa command. If VASA Provider service alreadybeen started, this step can be skipped.

IMPORTANT: Starting the VASA Provider requires super user privileges.

For example, showvasa and startvasa commands for enabling the VASA Provider:

cli% showvasa

-Service- ---------VASA_API2_URL------------- -MemUsage(MB)- -Version-Disabled https://s931.3pardata.com:9997/vasa 87 2.1.8

cli% startvasa

The VASA Provider Server will start shortly.

cli% showvasa

-Service- ---------VASA_API2_URL------------- -MemUsage(MB)- -Version-Enabled https://s931.3pardata.com:9997/vasa 87 2.1.8


Assigning 3PAR Administrative User for VVols and Setting Up Storage ContainerThe VASA Provider must be registered to the vCenter environment using an array credential.This user credential must be granted rights to access either the array or a single virtual domainon the array, and, in order to create VVols, this user must be granted edit rights.When registered, the 3PAR VASA Provider exposes a single VVol storage container, whichappears as a VVol Datastore in vSphere. The storage container is in the same 3PAR virtualdomain as the user that registers the VASA Provider.Before registering the VASA Provider with VMware vCenter, complete the following procedure:1. Determine whether VVol storage container should be separated into their own 3PAR virtual

domain. If so, create a 3PAR virtual domain by using the createdomain <domain>command.

NOTE:• For detailed info about 3PAR Virtual Domain, refer to 3PAR Command Line Interface

Administrator’s Manual.• The user that registered the VASA Provider can only see objects in CPGs which are

associated with the same virtual domain as the user. If the registered VASA Provideruser is part of the “all” domain and is not tied to a particular virtual domain, then theVASA Provider can only see objects that are not associated with any domain (ie. thedomain field is “-“). In this scenario, the CPGs and the storage LUNs and VVolsassociated with those CPGs in a particular domain are not visible to the VASA Provider,and thus not visible to the vCenter or associated ESX clients that registered this instanceof the VASA Provider. Also note, that when the VASA Provider is registered to a vCenter,it can only operate in one virtual domain for that vCenter. However different vCenterscan register to the VASA Provider, using different users associated with different virtualdomains.

2. Determine the 3PAR administrative user for registering the VASA Provider. In order for thevSphere environment to create VVols on the array, provide this user information to theVMware administrator for registration of the VASA Provider. If a virtual domain is used, thisuser must be assigned to the same domain as determined in step 1.

3. Grant this 3PAR administrative user the edit role and associate this user with the domainby using the createuser <user> <domain> edit command. For example:

cli% createuser vvoluser vvoldomain edit

4. Associate the 3PAR administrative user with the 3PAR domain that hosts the storagecontainer.

5. Define additional CPGs, with optional growth limits, for VVol usage within that virtual domain.

NOTE: VVol growth will only be limited if all CPGs in the same domain have growth limits.For more information, see “Allocate Common Provisioning Groups (CPG) for VVols on thearray” (page 108)). The virtual domain, or root domain (depending on which is used by theVASA Provider) must have at least one CPG, from which VVols can be provisioned.


Allocate Common Provisioning Groups (CPG) for VVols on the array

Specifying VVol Capability ProfilesCommon provisioning groups (CPGs) are used to expose 3PAR capabilities to the vSphereenvironment. On the 3PAR array, the implementation of VMware capability profiles is supportedusing the 3PAR Common Provisioning Group (CPG). CPGs allow the storage administrator todefine and control which storage capabilities should be exposed to the vSphere through thestorage container.

Setting Up Common Provisioning GroupsIn the VMware VVol data model, a Storage container is associated with different CapabilityProfiles. At least one capability profile needs to be created in the storage container (same domainas the VVset storage container) prior to VASA registration. Thus, at least one CPG must becreated. In a fresh installation, at least one CPG is automatically created.Use showcpg to show the existing common provisioning groups in the root domain.For example, showcpg command:

cli% showcpg

---------------(MB)--------------------Volumes---- -Usage- ---- Usr ---- -- Snp --- -- Adm ---

Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used0 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------4 total 5 0 660480 615424 0 0 0 0

Use the createcpg command to create a CPG in 3PAR array. You can specify different devicetype and RAID type settings with this command.For example, createcpg and showcpg commands:

cli% createcpg mycpgcli% showcpg


Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used5 mycpg - - 0 0 0 0 0 0 0 0 0 0 00 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------5 total 5 0 660480 615424 0 0 0 0


The Special vvol_ CPGA new interface for constraining which CPGs are used by VVols has been provided. If you wishto constrain the list of CPGs available to VVols, you can create new (or rename) CPGs andprovide the vvol_ prefix. When a default CPG is selected (if the storage profile does not providea Provisioning Group constraint), then only CPGs with the vvol_ prefix will be used.For example, a VM created without a storage profile, or Provisioning Group constraint, will beprovisioned to the vvol_ESX60 CPG as shown below.For example, special vvol_prefix CPG:

cli% showcpg


Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used5 mycpg - - 0 0 0 0 0 0 0 0 0 0 00 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0vvol_ESX60 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------6 total 5 0 660480 615424 0 0 0 0


Default 3PAR OS CPGs for VVolsThe vSphere administrator can choose array capability constraints when provisioning VMs inVVol storage containers. These capability constraints are known as storage profiles. With the3PAR OS, one of the constraints the vSphere administrator can select is the CPG. If the vSphereadministrator chooses not to create a storage profile or the storage profile does not haveconstraints for a CPG, the VASA Provider automatically selects a default CPG for the VVols ofthat VM.The VASA Provider uses the following algorithm to select CPGs for newly created VVols:

• If there are any CPG names that begin with vvol_, all other CPGs are eliminated fromconsideration for provisioning.

• If the space available to any of the CPGs is less than 10% of the total space originallyprovisioned for that CPG, these CPGs are eliminated from consideration for provisioning,unless all CPGs are under the same space pressure.

• Of the remaining CPGs under consideration, a balance of performance and availability isused to consider which CPG to provision the VVols. The order of CPGs selected dependson the CPG configuration drive type (NL, FC, SSD) and RAID (r0, r1, r5, r6) settings.The VASA Provider chooses the first CPG to match the preferred drive/raid order as follows:

SSDr57FCr51SSDr68FCr62SSDr19FCr13FCr010NLr64NLr011NLr15SSDr012NLr56

NOTE:• At least one CPG must be defined on the array and used in the associated VVol domain.

When no CPGs exist, registration of the VASA Provider fails and reports a Rescan Error onthe VMware vCenter Storage Provider page.

• VVol growth will only be limited if all CPGs in the same domain have growth limits.


Registering the 3PAR VASA Provider with VMware vCenterThe storage administrator must provide the VASA Provider URL and the 3PAR user/administratorcredentials to the VMware administrator for registration inside VMware vCenter. To discover theURL that needs to be provided to the VMware administrator, use the 3PAR CLI showvasacommand. To provide the VASA Provider SSL certificate thumbprint that allows the vSphereadministrator to validate the identity of the VASA Provider on the array, use the showvasa-cert command.For example, showvasa and showvasa -cert commands

cli% showvasa


cli% showvasa -cert

--------VASA_Cert_Subject-------- ----------------VASA_Cert_Thumbprint------------------------C=US/ST=CA/O=HP/OU=3PAR/CN=s558.3pardata.com 9C:FE:E9:BB:A6:B3:C6:76:2F:F1:CD:CB:AF:C3:39:44:BD:D8:1F:4B

CAUTION: Once the VASA Provider is registered with VMware vCenter, coordinate any changeof the VMware vCenter-registered VASA Provider user name or password with the VMwareadministrator. The VMware administrator must immediately re-register the VASA Provider inVMware vCenter or the services offered by VASA Provider will immediately cease.

NOTE:• Register the 3PAR VASA Provider with one VMware vCenter Server at a time.

• Before registering the VASA Provider with a new VMware vCenter Server, unregister theold VMware vCenter Server, and then reset the VASA Provider SSL certificate by using3PAR CLI setvasa -reset command.

• If registration fails, indicating that VMCA certificate registration fails, verify that the VMwarevCenter Server, ESXi, and 3PAR array clocks are in sync. Additionally, use the setvasa-reset command and retry.

• If registration fails, indicating the VASA Provider URL is invalid, verify that the VASA ProviderURL used during registration matches exactly the URL displayed by using the showvasacommand.


Registering the Storage Provider (3PAR VASA Provider) with VMware vCenter1. From VMware vCenter, navigate to Home→Hosts and Clusters→vCenter

Server→Manage→Storage Providers.2. Click + to add a registration.3. Fill in the requested information.

• Name—User-defined name to represent the array

• URL—URL copied from showvasa output on the array

• Username—Username for array login

• Password—User password for array login

Figure 32 Register storage provider on vCenter (3PAR VASA Provider on vCenter)

4. Verify that the Storage Provider is online.

Figure 33 Storage provider registered with vCenter


5. Add a VVol datastore (3PAR StoreServ Storage container) by using the following procedure:a. Navigate to Home→Storage→<datacenter>→Actions.b. Select Storage, and then select New Datastore from the Actions list. The New

Datastore window appears.

Figure 34 Add a VVol datastore

c. In the Location dialog box, enter the location information for the new VVol datastore,and then click Next.

Figure 35 Location of new VVol datastore


d. In the Type dialog box, select VVol as the Type, and then click Next.

Figure 36 Select the datastore type

e. In the Name and container selection dialog box, enter a Datastore name, select acontainer, and then click Next.

Figure 37 Datastore name and select container

f. In the Select hosts accessibility dialog box, select which hosts can access the VVolStorage Container, and then click Next.


g. In the Ready to complete dialog box, verify the settings, and then click Finish to makethe new VVol datastore available to VMware vCenter.

Figure 38 Verify and finish

IMPORTANT: Avoid using the storage administration command to manage or manipulateVVols on the array.

For more information about the following topics, see the VMware documentation:• Creating VMware virtual machines

• Managing VVols

• Creating storage policies for the VVol datastoreFor more information about viewing VVols as they exist on 3PAR StoreServ Storage, such asshowvvolvm and showvvolvm -vv, refer to theHPE 3PARCommand Line Interface Reference.For licensing information for VVols implementation with 3PAR StoreServ Storage, refer to thetechnical white-paper: Implementing VMware Virtual Volumes onHPE 3PARStoreServ (publication4AA5-6907ENW).


Additional Features with 3PAR OS 3.2.2Beginning with 3PAR OS 3.2.2, additional features have been added to the VMware VVolsupport, including:

• Support array level Adaptive Flash Cache as a VVol capabilityThis feature can be selected by a vSphere administrator if the storage administrator hasconfigured whole-array adaptive flash cache.

• Support multiple Storage Containers

• New CLI commands introduced: setvvolsc and showvvolsc

• Using a VVset as a storage containerUse the showvvset -summary command to see which VVset is defined to be a storagecontainer.

• Migration from the domain or “root” domain to a VVset storage container with the samename.

NOTE: Downgrade is not supported once migrated to a VVset storage container.

• Support multiple vCenter servers registering to the same VASA provider through servercertificate management mode.

NOTE: With the addition of multiple storage container support, the showvvolvm commandnow requires the use of the -sc option.

• Usability enhancements on setvasa and showvvolvm commands.

VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.2 Set upThis topic provides the necessary information to set up VVols on the 3PAR StoreServ Storage.

Setting up VVols on the 3PAR StoreServ Storage with 3PAR OS 3.2.21. “Managing Certificates” (page 116)2. “Enabling the 3PAR VASA Provider” (page 120)3. “Assigning 3PAR Administrative User for VMware VVols” (page 121)4. “Creating and Viewing Storage Containers” (page 123)5. “Allocating Common Provisioning Groups (CPG) for VVols on the Array” (page 124)6. “Enabling 3PAR Adaptive Flash Cache for use with VMware VVol (Optional)” (page 126)7. “Registering the VASA Provider (3PAR VASA Provider) with vCenter” (page 126)

Managing Certificates

VASA Provider Certificate ManagementThe 3PAR VASA provider uses a public key certificate to provide encrypted connections betweenvSphere and the VASA provider. The VASA 2.0 protocol allows for two different certificatemanagement modes, vSphere managed (a.k.a client-managed) and VASA provider managed(a.k.a server.managed).When using vSphere managed certificates, a CA (certificate authority) certificate associated withvCenter, known as the VMCA certificate, is used to sign the VASA provider’s certificate. vSpheretakes ownership of the VASA provider’s certificate and manages it from that point forward.When using VASA provider (server) managed certificates, the storage administrator managesthe VASA provider’s certificate. The VASA provider may use self-signed certificates, or use


certificates signed by a PKI (public key infrastructure) CA. Review the below to properly set upcertificate management to use one of these certificate management modes.The ability to choose certificate management mode was introduced in 3PAR OS version 3.2.2.Prior to this, only vSphere managed (client-managed) certificate mode was supported.Beginningwith 3PAR OS 3.2.2, certificate management mode defaults to VASA provider managed(server-managed) mode, unless the VASA provider had already been enabled on a prior versionof the 3PAR OS, in which case the mode does not change (remains client-managed).

Choosing the Certificate Management ModeThe ultimate decision on which VASA provider certificate management mode should be usedrests primarily on the need to connect more than one vCenter environment to array/VASA provider.When using client-managed certificate mode, the vCenter environment takes ownership of theVASA provider’s certificate, simplifying management of this certificate. However, when connectingtwo or more vCenter servers to the VASA provider, both cannot manage the certificate at thesame time. In this situation, the VASA provider must be set to server-managed certificate mode.Refer to the below topics for changing the certificate management mode.

WARNING! Changing the certificate management mode will disrupt any active VASAconnections. The VASA provider must be re-registered with any vCenters that are currentlyregistered to this VASA provider. Do not change certificate management modes withoutcoordinating with the vSphere administrator.

NOTE: When choosing client-managed certificate mode, the clocks on the array, vCenter andESXi hosts must be closely in-sync (less than 1 minute apart). Without this, the certificatesgenerated will be out of sync, and thus fail to support SSL communication. When usingserver-managed certificate mode, the clocks on vCenter, ESXi and the array must still be in sync,however the tolerances required are relaxed, since certificates do not need to be dynamicallygenerated during operation.

Server-Managed Certificate Management ModeServer-Managed certificate management mode allows multiple vCenter environments to connectto the VASA provider, and also allows the VASA provider to use self-signed certificates, orcertificates issued from a PKI. When using server-managed certificate management mode, thestorage administrator is responsible for managing the VASA provider’s certificate, includingrefreshing the certificate after expiration.

NOTE: The above procedure can be used to issue a new certificate to the VASA provider if acertificate needs to be replaced (such as when it expires.)

Using 3PAR VASA Provider with a Public Key Infrastructure (PKI)To integrate the VASA provider with a public key infrastructure the Storage Administrator mustgenerate a CSR (certificate signing request) on behalf of the VASA provider. This CSR must besent to the PKI’s certification authority (CA). The CA verifies and validates it by signing it with itsprivate key. This digital certificate is then imported to an array and the VASA provider’s certificatemanagement mode is set to server. The following process can be used to generate a CSR andimport the signed certificate into the array. This process also assumes, but does not require, theCA certificate used to sign the VASA provider has already been imported as a trusted certificateinto vSphere. For steps to add the CA root and intermediate certificate to vSphere, refer tovSphere security guide.

VMware VVol on 3PAR StoreServ Storage with 3PAR OS 3.2.2 Set up 117

1. Discover the array name or IP address.The information used to create the certificate signing request must match that generated bythe VASA provider. Use the showvasa command to discover the array’s hostname/IPaddress.Use showvasa to discover the array’s hostname/IP address:

cli% showvasa


2. Generate a CSR.A certificate signing request must accurately represent the identity of the VASA provider onthe array, as well as contain enough information about the requester such that the CAadministrator can identify the requester for proper validation.If the <name/IP> returned from showvasa is a DNS host name, then issue:

NOTE: A <keySize> of 1024 or 2048 can be used.

cli% createcert vasa -csr -keysize <keySize> -days <daysValid> -C <CountryCode> —ST <State/Province> —L <Locality/City>—O <OrganizationalUnit> —CN <name> —SAN DNS:<name>

If the <name/IP> returned from showvasa is an IP address then issue:

cli% createcert vasa -csr -keysize <keySize> -days <daysValid> -C <CountryCode> -ST <State/Province> -L <Locality/City>-O <Organization> -OU <OrganizationalUnit> -CN <name> -SAN IP:<IP>

3. Send the CSR to the Certificate Authority.The output from createcert must be captured and sent to the PKI’s certification authority.Different PKI infrastructures will have different procedures for this process. Refer to yourorganization’s PKI procedures to complete this step. Once you receive a signed certificateback, as well as a copy of the CAs certificate, this step is complete.

4. Import the Signed Certificate.Place the certificate signed by CA and the CA certificate into separate files. Then use theimportcert command to import them into the array. The certificate has to be encoded inthe PEM certificate format.

cli% importcert -service vasa <certifiFilename><CAcertFilename>

5. Change the VASA Providers Certificate Management Mode (if needed).Change 3PAR VASA provider’s certification management to server mode. This step is notrequired if the VASA provider is already in server mode. Use showvasa -cert to discoverthe current setting.

cli% setvasa -certmgmt server


Using 3PAR VASA Provider with a Self-Signed CertificateThe following process can be used by the storage administrator to generate a self-signed certificatefor use by VASA provider in server-managed certificate mode. This option allows multiple vCenterclients to connect to VASA provider.1. Discover the array name or IP address.

The information used to create the certificate signing request must match that generated bythe VASA provider. Use the showvasa command to discover the array’s hostname/IPaddress.

cli% showvasa

-Service- -----------VASA_API2_URL----------- -MemUsage(MB)- -Version-Enabled https://<name/IP>:9997/vasa 726 2.2.9

2. Generate a self-signed certificate.A certificate signing request must accurately represent the identity of the VASA provider onthe array, as well as contain enough information about the requester such that the CAadministrator can identify the requester for proper validation.If the <name/IP> returned from showvasa is a DNS host name, then issue:

cli% createcert vasa -selfsigned -keysize <keySize> -days <daysValid> -C <CountryCode> -ST <State/Province> -L<Locality/City> -O <Organization> -OU <OrganizationalUnit> -CN <name> -SAN DNS:<name>

If the <name/IP> returned from showvasa is an IP address then issue:

NOTE: A <keySize> of 1024 or 2048 can be used.

cli% createcert vasa -selfsigned -keysize <keySize> -days <daysValid> -C <CountryCode> -ST <State/Province> -L<Locality/City> -O <Organization> -OU <OrganizationalUnit> -CN <name> -SAN IP:<IP>

3. Change the VASA Provider’s Certificate Management Mode (if needed). Change 3PARVASA provider’s certification management to “server” mode. This step is not required if theVASA provider is already in “server” mode. Use showvasa -cert to discover the currentsetting.

cli% setvasa –certmgmt server

Client-Managed Certificate Management ModeClient-Managed certificate management mode eliminates the need for the storage administratorto manage the VASA provider’s certificate. However, in this mode, only a single vCenterenvironment can connect to the VASA provider at any one time. Registering a second vCenterenvironment to the VASA provider will cause the original vCenter environment to lose allcommunication with the VASA provider, and thus no longer be able to discover array information,nor manage previously created VMs in VVol storage containers.


Using 3PAR VASA Provider with vSphere-Managed Certificate (VMCA)In this option, vSphere acts as a PKI certificate authority. During registration and the initialconnection to VASA Provider, vSphere requests the VASA provider to generate a certificatesigning request (CSR). vSphere then returns the VMCA-signed certificate back to VASA Provider.All existing connections to the VASA provider are closed and new connections are establishedusing the new certificate.In order to use the VASA provider with a vSphere-Managed certificate, the VASA provider’scertificate management mode must be set to “client.”

Changing Server Certificate Management to Client Certificate Management1. Examine the date and time on the array, ESX hosts and vCenter host. to determine if the

clocks are in sync. Use showdate to determine the current time on the array. Refer tosetnet to determine how to set up ntp on the array, if required.

2. Determine the current certificate management mode:showvasa –cert

3. If the VASA provider is not already in client certificate management mode, issue:setvasa –certmgmt client

4. To assure the VASA provider has an up-to-date self-signed certificate, issue:setvasa -reset

5. The VASA Provider can now be registered with a vCenter.

NOTE: For 3PAR OS 3.2.2 new installation, by default, the preferred certification managementmode is “server managed” mode.

Enabling the 3PAR VASA ProviderThe 3PAR VASA Provider enables communication between the vCenter and the 3PAR StoreServStorage array management interface by using the VMware VASA 2.0 Protocol.The VASA Provider Service is automatically started and enabled on the 3PAR array. Theshowvasa command can be used to display the current status of the service. The VASA ProviderService can also be started and enabled manually using the startvasa command as shownin following example:

IMPORTANT: Starting the VASA Provider requires super user privileges.

For example, showvasa and startvasa commands for enabling the VASA Provider:

cli% showvasa

-Service- ---------VASA_API2_URL------------- -MemUsage(MB)- -Version-Disabled https://s931.3pardata.com:9997/vasa 87 2.1.8

cli% startvasa

The VASA Provider Server will start shortly.

cli% showvasa



Assigning 3PAR Administrative User for VMware VVolsThe VASA Provider must be registered to the vCenter environment using an array credential.This user credential must be granted rights to access either the array or a single virtual domainon the array, and, in order to create VVols, this user must be granted edit rights. When the VASAprovider is configured to use a self-signed certificate, or a certificate signed by a PKI, the VASAprovider can be registered with multiple vCenter environments simultaneously. In this case, thestorage administrator can choose to have these multiple vCenter environments registered tofunction within a single virtual domain (or no domain) or split across multiple Virtual Domains.For each virtual domain, a separate 3PAR administrative user must be created that has beenassigned to that virtual domain. When the vCenter environment is registered using that user, allstorage objects visible to the vCenter environment will be limited to objects within that virtualdomain.When registered, the 3PAR VASA Provider exposes all VVol storage containers that have beencreated in that virutal domain. VVol storage containers appear as VVol Datastores in vSphere.All storage containers exposed to vSphere will be in the same 3PAR virtual domain as the userthat registers the VASA Provider.Before registering the VASA Provider with VMware vCenter, complete the following procedure:1. Determine if you have multiple vCenter servers to register to the array. Multiple vCenter

servers can either be separated into their own virtual domains or they can all share the samedomain. All vCenter servers in the same virtual domain will see the same VVol storagecontainers.A 3PAR virtual domain can be created using the 3PAR OS CLI createdomain <domain>command.

NOTE:• Refer to HPE 3PAR Command Line Interface Administrator’s Manual for detailed info

about 3PAR Virtual Domain.• The user that registered the VASA Provider can only see objects in CPGs which are

associated with the same virtual domain as the user. If the registered VASA Provideruser is part of the “all” domain and is not tied to a particular virtual domain, then theVASA Provider can only see objects that are not associated with any domain (ie. thedomain field is “-“). In this scenario, the CPGs and the storage LUNs and VVolsassociated with those CPGs in a particular domain are not visible to the VASA Provider,and thus not visible to the vCenter or associated ESX clients that registered this instanceof the VASA Provider. Also note, that when the VASA Provider is registered to a vCenter,it can only operate in one virtual domain for that vCenter. However different vCenterscan register to the VASA Provider, using different users associated with different virtualdomains.

2. Determine the 3PAR administrative user for registering the VASA Provider. In order for thevSphere environment to create VVols on the array, provide this user information to theVMware administrator for registration of the VASA Provider. If a virtual domain is used, thisuser must be assigned to the same domain as determined in step 1.

3. Grant this 3PAR administrative user the edit role and associate this user with the domainby using the createuser <user> <domain> edit command. For example:

cli% createuser vvoluser vvoldomain edit

4. Associate the 3PAR administrative user with the 3PAR virtual domain that hosts the storagecontainer.


5. Define additional CPGs, with optional growth limits, for VVol usage within that virtual domain.Note that all VVol storage containers in the same domain will share CPGs within that domain.

NOTE: VVol growth will only be limited if all CPGs in the same domain have growth limits.For more information, see “Allocate Common Provisioning Groups (CPG) for VVols on thearray” (page 108). The virtual domain, or root domain (depending on which is used by theVASA Provider) must have at least one CPG, from which VVols can be provisioned.

6. Repeat step 2 through step 4 for each Virtual Domain created in step 1, and then continueto the next topic, “Creating and Viewing Storage Containers” (page 123), to create VVolstorage containers for those domains.


Creating and Viewing Storage ContainersWith 3PAROS 3.2.2, the ability to create multiple storage containers is added. Storage containeris now a VVset with a special flag. You can determine if you already have VVol storage containersby using the showvvolsc command. If you have migrated VVols from 3.2.1, VVset storagecontainer objects will automatically be created upon migration to 3.2.2 or later. If you need tocreate a storage container, first use createvvset to create an empty VVset, then usesetvvolsc -create set:<vvsetname> to assign the storage container flag to the VVset.Refer to the cli syntax for these commands. More than one storage container is supported in thisversion.If you do not see any storage containers, you must create one as follows:1. Use the createvvset command:

cli% createvvset -domain <domain> <vvset_name>

2. Use the setvvolsc command:

cli% setvvolsc -create set:<vvset_name>

Setting set:newvvset as a VVol storage container.

There are more options in the showvvolsc command to see additional details.For example, showvvolsc and createvvset commands:

cli% showvvolsc

-------(MB)-------Name Num_VMs Num_VVols In_Use Provisioned------------------------------------------total 0 0 0 0

cli% createvvset cust1vvol

cli% setvvolsc -create set:cust1vvolSetting set:cust1vvol as a VVol storage container

cli% createvvset cust2vvol

cli% setvvolsc -create set:cust2vvolSetting set:cust2vvol as a VVol storage container

cli% showvvolsc

-------(MB)-------Name Num_VMs Num_VVols In_Use Provisionedcust1vvol 0 0 0 0cust2vvol 0 0 0 0------------------------------------------total 0 0 0 0

NOTE: Downgrade from 3PAR OS 3.2.2 to OS 3.2.1 is not supported once migrated to aVVset storage container.VASA Provider remains with vSphere managed certificate mode (client-managed) whenupgrading from 3PAR OS 3.2.1 to OS 3.2.2. See “Managing Certificates” (page 116) fordetailed info on how to change the VASA Provider’s certification management to servermode.


Allocating Common Provisioning Groups (CPG) for VVols on the Array

Specifying VVol Capability ProfilesCommon provisioning groups (CPGs) are used to expose 3PAR capabilities to the vSphereenvironment. On the 3PAR array, the implementation of VMware capability profiles is supportedusing the 3PAR Common Provisioning Group (CPG). CPGs allow the storage administrator todefine and control which storage capabilities should be exposed to the vSphere through thestorage container.

Setting Up Common Provisioning GroupsIn the VMware VVol data model, a Storage container is associated with different CapabilityProfiles. At least one capability profile needs to be created in the storage container (same domainas the VVset storage container) prior to VASA registration. Thus, at least one CPG must becreated. In a fresh installation, at least one CPG is automatically created.Use showcpg to show the existing common provisioning groups in the root domain.For example, showcpg command:

cli% showcpg


Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used0 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------4 total 5 0 660480 615424 0 0 0 0

Use the createcpg command to create a CPG in 3PAR array. You can specify different devicetype and RAID type settings with this command.For example, createcpg and showcpg commands:

cli% createcpg mycpgcli% showcpg


Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used5 mycpg - - 0 0 0 0 0 0 0 0 0 0 00 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------5 total 5 0 660480 615424 0 0 0 0


The Special vvol_ CPGA new interface for constraining which CPGs are used by VVols has been provided. If you wishto constrain the list of CPGs available to VVols, you can create new (or rename) CPGs andprovide the vvol_ prefix. When a default CPG is selected (if the storage profile does not providea Provisioning Group constraint), then only CPGs with the vvol_ prefix will be used.For example, a VM created without a storage profile, or Provisioning Group constraint, will beprovisioned to the vvol_ESX60 CPG as shown below.For example, special vvol_prefix CPG:

cli% showcpg


Id Name Domain Warn% VVs TPVVs TDVVs Usr Snp Total Used Total Used Total Used5 mycpg - - 0 0 0 0 0 0 0 0 0 0 00 FC_r1 - - 0 0 0 0 0 0 0 0 0 0 01 FC_r5 - - 5 0 0 5 0 660480 615424 0 0 0 02 FC_r6 - - 0 0 0 0 0 0 0 0 0 0 03 NL_r6 - - 0 0 0 0 0 0 0 0 0 0 0vvol_ESX60 - - 0 0 0 0 0 0 0 0 0 0 0---------------------------------------------------------------------------------6 total 5 0 660480 615424 0 0 0 0

Default 3PAR OS CPGs for VVolsThe vSphere administrator can choose array capability constraints when provisioning VMs inVVol storage containers. These capability constraints are known as storage profiles. With 3PAROS, one of the constraints the vSphere administrator can select is the CPG. If the vSphereadministrator chooses not to create a storage profile or the storage profile does not haveconstraints for a CPG, the VASA Provider automatically selects a default CPG for the VVols ofthat VM.The VASA Provider uses the following algorithm to select CPGs for newly created VVols:

• If there are any CPG names that begin with vvol_, all other CPGs are eliminated fromconsideration for provisioning.

• If the space available to any of the CPGs is less than 10% of the total space originallyprovisioned for that CPG, these CPGs are eliminated from consideration for provisioning,unless all CPGs are under the same space pressure.

• Of the remaining CPGs under consideration, a balance of performance and availability isused to consider which CPG to provision the VVols. The order of CPGs selected dependson the CPG configuration drive type (NL, FC, SSD) and RAID (r0, r1, r5, r6) settings.The VASA Provider chooses the first CPG to match the preferred drive/raid order as follows:

SSDr57FCr51SSDr68FCr62SSDr19FCr13FCr010NLr64NLr011NLr15SSDr012NLr56


Enabling 3PAR Adaptive Flash Cache for use with VMware VVol (Optional)Adaptive Flash Cache provides DRAM cache extension by leveraging 3PAR first in classvirtualization technologies. This functionality allows dedicating a portion of SSDs capacity as anaugmentation of the 3PAR Array primary cache, reducing application response time for readintensive I/O workloads and improving write throughout in mixed workload environments.The flash cache needs to be enabled/disabled for the entire system for VASA Provider to exposeit as a capability to the vSphere.1. Create the flash cache in the StoreServ Storage system:

createflashcache <size>g|G|t|T

2. Enable flash cache for all volumes in the system:setflashcache enable sys:all

orDisable flash cache:setflashcache disable sys:all

Registering the VASA Provider (3PAR VASA Provider) with vCenterThe storage administrator must provide the VASA Provider URL and the 3PAR user/administratorcredentials to the VMware administrator for registration inside VMware vCenter. To discover theURL that needs to be provided to the VMware administrator, use the 3PAR CLI showvasacommand.To provide the VASA Provider SSL certificate thumbprint that allows the vSphere administratorto validate the identity of the VASA Provider on the array, use the showvasa -cert command.For example, showvasa and showvasa -cert commands:

cli% showvasa


cli% showvasa -cert

Mgmt_Mode -----------------------Subject-------------- ----------------Fingerprint------------------------client C=US/ST=CA/O=HP/OU=3PAR/CN=s293.3pardata.com 9C:FE:E9:BB:A6:B3:C6:76:2F:F1:CD:CB:AF:C3:39:44:BD:D8:1F:4B

CAUTION: Once the VASA Provider is registered with vCenter, coordinate any change of thevCenter-registered VASA Provider user name or password with the VMware administrator. TheVMware administrator must immediately re-register the VASA Provider in VMware vCenter orthe services offered by the VASA Provider will immediately cease.

NOTE:• If registration fails, indicating that VMCA certificate registration fails, verify that the vCenter

Server, ESXi, and 3PAR array clocks are in sync. Additionally, use the setvasa -resetcommand and retry.

• If registration fails, indicating the VASA Provider URL is invalid, verify that the VASA ProviderURL used during registration matches exactly the URL displayed by using the showvasacommand.


Registering the Storage Provider (3PAR VASA Provider) with vCenter1. From vCenter, navigate to Home→Hosts and Clusters→vCenter

Server→Manage→Storage Providers.2. Click + to add a registration.3. Fill in the requested information.

• Name—User-defined name to represent the array

• URL—URL copied from showvasa output on the array

• Username—Username for array login

• Password—User password for array login

Figure 39 Registering storage provider on vCenter (3PAR VASA Provider on vCenter)

4. Verify that the Storage Provider is online.

Figure 40 Example Storage provider registered in the vCenter


5. Add a VVol datastore (3PAR StoreServ Storage container) by using the following procedure:a. Navigate to Home→Storage→<datacenter>→Actions.b. Select Storage, and then select New Datastore from the Actions list. The New

Datastore window appears.

Figure 41 Add a VVol datastore

c. In the Location dialog box, enter the location information for the new VVol datastore,and then click Next.

Figure 42 Location of new VVol datastore


d. In the Type dialog box, select VVol as the Type, and then click Next.

Figure 43 Select the datastore type

e. In the Name and container selection dialog box, enter a Datastore name, select astorage container, and then click Next.

Figure 44 Name and container selection

f. In the Select hosts accessibility dialog box, select which hosts can access the VVolStorage Container, and then click Next.


g. In the Ready to complete dialog box, verify the settings, and then click Finish to makethe new VVol datastore available to vCenter.

Figure 45 Verify and finish

IMPORTANT: Avoid using the storage administration commands to manage ormanipulate VVols on the array.

For more information about the following topics, see the VMware documentation:• Creating VMware virtual machines

• Managing VVols

• Creating storage policies for the VVol datastoreFor more information about viewing VVols as they exist on 3PAR StoreServ Storage, such asshowvvolvm, showvvolvm -vv, showvvolvm -sc, or showvvset -summary, refer to the3PAR Command Line Interface Reference.For licensing information for VVols implementation with 3PAR StoreServ Storage, refer to thetechnical white-paper: Implementing VMware Virtual Volumes onHPE 3PARStoreServ (publication4AA5-6907ENW).


11 Configuring for 3PAR Peer PersistenceThis topic describe the requirements to set up a VMware ESX host to use 3PAR Peer Persistenceprovided by 3PAR StoreServ Storage.

3PAR Peer PersistenceFor complete description of the 3PAR Peer Persistence feature and how to configure 3PARStoreServ Storage for 3PAR Peer Persistence with Automatic Transparent Failover and ManualTransparent Switchover, refer to theHPE 3PARRemote Copy Software User Guide at the HewlettPackard Enterprise Storage Information Library website.For information about supported 3PAR Peer Persistence configurations for VMware andexceptions, see the 3PAR Peer Persistence support matrix in the SPOCK website (SPOCKHome→Software→Array SW: 3PAR).For information about supported hardware and software platforms, refer the HPE 3PAR StoreServStorage Interoperability for VMware on the SPOCK website (SPOCK Home→Explore StorageInteroperability With SPOCK→Explore HPE 3PAR StoreServ Storage interoperability).

NOTE:• Host VMware Persona 11 which enables asymmetric logical unit access (ALUA) is required

for VMware ESX hosts configured as part of 3PAR Peer Persistence configurations.• Configuration of VMware ESX hosts for 3PAR Peer Persistence should follow this HPE

3PAR VMware ESX/ESXi Implementation Guide.

More informationhttp://www.hpe.com/info/storage/docshttp://www.hpe.com/storage/spock

3PAR Peer Persistence 131



12 Host Persona MigrationThis topic provides guidance for successfully migrating your VMware ESX/ESXi hosts on 3PARStoreServ Storage from host Generic-Legacy Persona 6 to host VMware Persona 11 to ensurea seamless 3PAR host persona transition.

Host Persona OverviewChanging from host Generic-Legacy Persona 6 to host VMware Persona 11 for an ESX/ESXihost requires that there are no active or online paths for the ESX/ESXi host to the 3PAR StoreServStorage system.

NOTE: Target-ESX/ESXi host refers to the ESX/ESXi host on which the host persona is beingchanged.Temporary-ESX/ESXi refers to the ESX/ESXi host used to temporarily host the virtual machinesfrom the Target-ESX/ESXi.

Methods for Changing Host PersonaThere are two methods that can be used to complete the persona change for each ESX/ESXihost.• Online Process

Using VMware vMotion technology, the virtual machines on the selected Target-ESX/ESXihost are first migrated to a Temporary-ESX/ESXi host in a High Availability (HA) clusterenvironment sharing the same 3PAR StoreServ Storage system. The Target-ESX/ESXi hostis then powered off, while virtual machines continue to have access to 3PAR StoreServStorage system via the Temporary-ESX/ESXi host in the cluster. The requisite host personachange is made for the Target-ESX/ESXi host definition on the 3PAR StoreServ Storagesystem. After the change, the Target-ESX/ESXi host is powered up and all of its virtualmachines are restored back from the Temporary-ESX/ESXi host using VMware vMotion.When you use this method, during the migration process there will be a mix of hostGeneric-Legacy Persona 6 and host VMware Persona 11 based hosts in the cluster.ESX/ESXi clusters with heterogeneous host persona definitions are supported for thismigration process.

• Offline ProcessThe offline process requires the Target-ESX/ESXi host and any associated active virtualmachines to be shutdown. This approach is to power off the Target-ESX/ESXi host and anyassociated virtual machines for the host. The requisite host persona change is then madefor the Target-ESX/ESXi host definition on the 3PAR StoreServ Storage system. After thechange, the Target-ESX/ESXi host and its virtual machines are restarted.

132 Host Persona Migration

Preparing for Host Persona MigrationBefore beginning the migration process, you must follow these requirements:• The VMware ESX/ESXi configuration on the 3PAR StoreServ Storage for the

Target-ESX/ESXi host must follow the guidelines found in this guide.• Beginning with 3PAR OS 3.1.2, host VMware Persona 11 is supported. The host persona

migration requires a 3PAR StoreServ Storage with minimum array firmware version of 3PAROS 3.1.2. Any VMware ESX/ESXi configurations before 3PAR OS 3.1.2 must first upgradeto 3PAR OS 3.1.2 or later before performing this migration. For upgrade details, see theHPE 3PAR Operating System Upgrade Pre-planning Guide on the Hewlett PackardEnterprise Storage Information Library website.

• Remote Copy host configurations with independent ESX/ESXi hosts attached to the primaryand secondary arrays with 3PAR OS 3.1.2 or earlier 3PAR OS, host Generic-Legacy Persona6 is required. 3PAR StoreServ Storage systems in these configurations should be firstupgraded to 3PAR OS 3.1.3 prior to the persona migration of the attached ESX/ESXi host(s)from host Generic-Legacy Persona 6 to host VMware Persona 11.


Host Persona Pre-migrationOn the target-ESX/ESXi host, remove any current or existing custom SATP rules used with hostGeneric-Legacy Persona 6 on the 3PAR StoreServ Storage system.

Finding 3PAR Custom SATP RuleRun the following command to find any existing SATP custom rules and associated parameters.For ESXi 5.x:


For ESX/ESXi 4.x:


If 3PAR SATP custom rule does not exist, no value is returned by the command. Continue withthe migration.Example where 3PAR Custom SATP Rules exist:


#VMW_SATP_DEFAULT_AA 3PARdata VV user tpgs_off VMW_PSP_RR iops=1 HP 3PAR Custom iSCSI/FC/FCoE Rule

If 3PAR SATP custom rules exist, you must remove them before continuing the migration. See“Removing 3PAR Custom SATP Rules” (page 134)

Preparing for Host Persona Migration 133


Removing 3PAR Custom SATP RulesIf 3PAR custom SATP rules for use with host Generic-Legacy Persona 6 exists, run the followingcommand to remove them.With ESXi 5.x:

# esxcli storage nmp satp rule remove -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>

ESXi 5.x example with parameters:

# esxcli storage nmp satp rule remove -s "VMW_SATP_DEFAULT_AA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_off" -V "3PARdata" -M"VV" -e" HP 3PAR Custom iSCSI/FC/FCoE Rule"

With ESX/ESXi 4.x:

# esxcli nmp satp deleterule -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>

Migrating Using Online MethodThe following steps must be applied for each ESX/ESXi host having its host persona migrated.1. Login to the vSphere Client that manages the Target-ESX/ESXi host.2. Using VMware vMotion technology, vMotion all virtual machines from the Target-ESX/ESXi

host to a Temporary-ESX/ESXi host in the cluster. Verify the vMotion migration completessuccessfully before continuing to the next step.

3. If you have not already done so, on the Target-ESX/ESXi host, remove any current or existingcustom SATP rules used with host Generic-Legacy Persona 6 on the 3PAR StoreServStorage system. See “Finding 3PAR Custom SATP Rule” (page 133).

4. Add the 3PAR custom SATP rule for use with host VMware Persona 11:With ESXi 5.x:

# esxcli storage nmp satp rule add -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>


# esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV"-e "HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule"


With ESX/ESXi 4.x:

# esxcli nmp satp addrule -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>

ESX/ESXi 4.x example with parameters:

# esxcli nmp satp addrule -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PARCustom iSCSI/FC/FCoE ALUA Rule"

NOTE: For detailed information, see “Configure ESX/ESXi Multipathing for Round Robinvia SATP PSP” (page 42).

5. From the vSphere Client, place the Target-ESX/ESXi host in Maintenance Mode and shutdown the Target-ESX/ESXi host.

6. On the 3PAR array, run the 3PAR CLI sethost command to change the host persona to11 for the Target-ESX/ESXi host:

# sethost -persona 11 <esx-hostname>

Verify that the change was successful with the showhost command:

# showhost -persona <esx-hostname>

7. Restart the Target-ESX/ESXi host, and then run the showhost command to verify that theESX/ESXi host is reconnected.

# showhost <esx-hostname>

8. From the vSphere Client, exit the Target-ESX/ESXi host from Maintenance Mode. Verifythat the Target-ESX/ESXi host reconnected to the cluster successfully.

9. From vSphere Client, vMotion all virtual machines back from the Temporary-ESX/ESXi hostto the Target-ESX/ESXi host.

10. Verify the virtual machines run properly and the target devices have been claimed properlyby the SATP rule as expected.

11. Repeat the above procedure for each ESX/ESXi host until all ESX/ESXi hosts in the clusterare migrated to host VMware Persona 11.

Migrating Using Online Method 135

Migrating Using Offline MethodThe following steps must be applied for each ESX/ESXi host having its host persona migrated.1. On the Target-ESX/ESXi host, remove any current or existing custom SATP rules being

used with host Generic-Legacy Persona 6 on the 3PAR StoreServ Storage system. See“Finding 3PAR Custom SATP Rule” (page 133) and “Removing 3PAR Custom SATP Rules”(page 134).

2. Run the following command to add the Hewlett Packard Enterprise recommended 3PARcustom SATP rule for use with host VMware Persona 11:With ESXi 5.x:

# esxcli storage nmp satp rule add -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>


# esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV"-e "HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule"

With ESX/ESXi 4.x:

# esxcli nmp satp addrule -s <satp> -P <psp> -O <psp_option> -c <claim_option> -V <vendor> -M <model>

ESX/ESXi 4.x example with parameters:

# esxcli nmp satp addrule -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e"HP 3PAR Custom iSCSI/FC/FCoE ALUA Rule"

NOTE: For detailed information, see “Configure ESX/ESXi Multipathing for Round Robinvia SATP PSP” (page 42).

3. Shutdown the Target-ESX/ESXi host and its virtual machines.4. On the 3PAR array, run the 3PAR CLI sethost command to change the host persona to

11 for the Target-ESX/ESXi host:

# sethost -persona 11 <esx-hostname>

Verify that the change was successful with the showhost command:

# showhost -persona <esx-hostname>

5. Restart the Target-ESX/ESXi host, and then run the showhost command to verify that theESX/ESXi host is reconnected.

# showhost <esx-hostname>

6. Verify the virtual machines run properly and the target devices have been claimed properlyby the SATP rule as expected.

7. Repeat the above procedure for each ESX/ESXi host until all ESX/ESXi hosts in the clusterare migrated to host VMware Persona 11.


13 Support and other resourcesAccessing Hewlett Packard Enterprise Support

• For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website:www.hpe.com/assistance

• To access documentation and support services, go to the Hewlett Packard Enterprise SupportCenter website:www.hpe.com/support/hpesc

Information to collect• Technical support registration number (if applicable)

• Product name, model or version, and serial number

• Operating system name and version

• Firmware version

• Error messages

• Product-specific reports and logs

• Add-on products or components

• Third-party products or components

Accessing updates• Some software products provide a mechanism for accessing software updates through the

product interface. Review your product documentation to identify the recommended softwareupdate method.

• To download product updates, go to either of the following:

Hewlett Packard Enterprise Support Center Get connected with updates page:www.hpe.com/support/e-updates

◦

◦ Software Depot website:www.hpe.com/support/softwaredepot

• To view and update your entitlements, and to link your contracts and warranties with yourprofile, go to the Hewlett Packard Enterprise Support Center More Information on Accessto Support Materials page:www.hpe.com/support/AccessToSupportMaterials

IMPORTANT: Access to some updates might require product entitlement when accessedthrough the Hewlett Packard Enterprise Support Center. You must have an HP Passportset up with relevant entitlements.

Accessing Hewlett Packard Enterprise Support 137

http://www.hpe.com/assistance


http://www.hpe.com/support/e-updates


http://www.hpe.com/support/AccessToSupportMaterials

Websites

LinkWebsite

www.hpe.com/info/enterprise/docsHewlett Packard Enterprise Information Library

www.hpe.com/support/hpescHewlett Packard Enterprise Support Center

www.hpe.com/assistanceContact Hewlett Packard Enterprise Worldwide

www.hpe.com/support/e-updatesSubscription Service/Support Alerts

www.hpe.com/support/softwaredepotSoftware Depot

www.hpe.com/support/selfrepairCustomer Self Repair

www.hpe.com/info/insightremotesupport/docsInsight Remote Support

www.hpe.com/info/hpux-serviceguard-docsServiceguard Solutions for HP-UX

www.hpe.com/storage/spockSingle Point of Connectivity Knowledge (SPOCK) Storagecompatibility matrix

www.hpe.com/storage/whitepapersStorage white papers and analyst reports

Customer self repairHewlett Packard Enterprise customer self repair (CSR) programs allow you to repair your product.If a CSR part needs to be replaced, it will be shipped directly to you so that you can install it atyour convenience. Some parts do not qualify for CSR. Your Hewlett Packard Enterprise authorizedservice provider will determine whether a repair can be accomplished by CSR.For more information about CSR, contact your local service provider or go to the CSR website:www.hpe.com/support/selfrepair

Remote supportRemote support is available with supported devices as part of your warranty or contractual supportagreement. It provides intelligent event diagnosis, and automatic, secure submission of hardwareevent notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolutionbased on your product’s service level. Hewlett Packard Enterprise strongly recommends thatyou register your device for remote support.For more information and device support details, go to the following website:www.hpe.com/info/insightremotesupport/docs

Documentation feedbackHewlett Packard Enterprise is committed to providing documentation that meets your needs. Tohelp us improve the documentation, send any errors, suggestions, or comments to DocumentationFeedback ([email protected]). When submitting your feedback, include the documenttitle, part number, edition, and publication date located on the front cover of the document. Foronline help content, include the product name, product version, help edition, and publication datelocated on the legal notices page.

138 Support and other resources

http://www.hpe.com/info/enterprise/docs


http://www.hpe.com/assistance

http://www.hpe.com/support/e-updates


http://www.hpe.com/support/selfrepair

http://www.hpe.com/info/insightremotesupport/docs

http://www.hpe.com/info/hpux-serviceguard-docs


http://www.hpe.com/storage/whitepapers

http://www.hpe.com/support/selfrepair

http://www.hpe.com/info/insightremotesupport/docs

mailto:[email protected]

Index

Symbols3PAR Persistent Ports

for FCoE, 343PAR VASA Provider

enable, 106, 120registering with VMware vCenter, 111

Aaccessing

updates, 137

Cchallenge-handshake authentication protocol

configuring, 81configuring on host for iSCSI, 82setting up, 81

CHAP see challenge-handshake authentication protocolcommon provisioning groups see CPGsconfiguring

for FC, 9for FCoE, 28Peer Persistence, 131ports, 9

contacting Hewlett Packard Enterprise, 137CPGs

using by storage administrator, 108creating

storage, 89TDVVs, 89VVs, 90

customer self repair, 138

Ddata duplication, 89deduplication, 89deploying

Virtual Connect Direct-Attach Fibre Channel, 9documentation, 8

providing feedback on, 138drivers

Brocade, 9HBA, 9

Ffabric

setting up for FC, 15zoning for FC, 15

failover, 9iSCSI considerations, 88multipath considerations, 38

FCconfiguring, 9configuring round robin multipathing, 40configuring the host, 35creating host definition, 12Fibre Channel, 8

guidelines for FC switch vendors, 17handling SCSI queue full and busy messages, 49installing the drivers, 35installing the HBA, 35installing VM guest OS, 36multipath considerations and I/O load balancing, 38performance considerations for multiple host

configurations, 49persistent port setup, 20Persistent Ports, 19Priority Optimization, 19setting up fabric, 15setting up ports, 10target port limits, 18target port specifications, 18zoning fabric, 15

FCoE3PAR Persistent Ports, 343PAR Persistent Ports connectivity, 343PAR Persistent Ports setup, 343PAR Priority Optimization, 33configuring, 28creating the host definition, 30Fibre Channel over Ethernet, 8setting up FCoE initiator, 28setting up FCoE swtich, 28setting up FCoE target port, 28target port limits and specifications, 33using system BIOS to configure FCoE, 58

features3PAR Express Writes, 203PAR Persistent Ports for FC, 193PAR Persistent Ports for FCoE, 343PAR Priority Optimization for FC, 193PAR Priority Optimization for FCoE, 333PAR VAAI Plug-in 1.1.1 for ESX/ESXi 4.1, 523PAR VAAI Plug-in 2.2.0 for ESXi 5.x and ESXi 6.0,

523PAR VASA Provider, 106, 120for FC ESX/ESXi 4.1, ESXi 5.x, and ESXi 6.0, 51out-of-space condition, 54Peer Persistence, 131Persistent Checksum over FC, 20Persistent Ports for iSCSI, 27Priority Optimization for iSCSI, 26Smart SAN, 16storage I/O control, 51summary, 56TP LUN Reporting, 56UNMAP storage primitive for space reclaim, 53VAAI, 51Virtual Connect Direct-Attach Fibre Channel, 9VVols, 99

Fibre Channel see FCFibre Channel over Ethernet see FCoEfirmware

Brocade, 9

139

Gguest OSs, 36

installing, 87

HHardware iSCSI

independent, 83HBA

Host Bus Adapter, 7host

booting, 98configuring as an FC connection, 35configuring as an FCoE initiator connecting to an FC

target or FCoE target, 58configuring CHAP, 82configuring FCoE switch, 58configuring for an iSCSI connection, 64configuring initiator FCoE to FC target, 61configuring initiator FCoE to FCoE target, 63creating definition for FC, 12creating definition for FCoE, 30creating definition for iSCSI, 24multiple host configurations for FC, 49setting up the iSCSI initiator, 65setting up the switch, 65

host persona1, 1411, 24, 31, 103, 1312, 14Generic-Legacy Persona 6, 12, 14, 24, 30, 32, 41, 43,

44, 45, 47, 132, 133, 134methods for changing, 132migrating using offline method, 136migrating using online method, 134migration, 132overview, 132pre-migration, 133preparing for migration, 133removing custom SATP rules, 134VMware Persona 11, 12, 13, 14, 24, 30, 32, 41, 43,

44, 45, 47, 132, 133, 134, 136

II/O path policy

round robin, 40IPv4, 23IPv6

iSCSI addressing, 22vLAN tagging, 22

iSCSIadditional features, 88configuring, 21configuring a service console connection for iSCSI

storage, 71configuring CHAP on host, 82configuring host for an iSCSI connection, 64configuring the Software iSCSI initiator, 76considerations for multiple host configurations, 88

creating a VMkernel port for Software iSCSI ESX 4.0,68

creating host definition, 24Enterprise, 64failover considerations, 88Hardware iSCSI support, 83installing Software iSCSI, 65installing VM guest OS, 87IPv6 addressing, 22IPv6 vLAN tagging, 22multipath load balancing, 88Persistent Ports, 27Priority Optimization, 26setting up persistent port, 27setting up ports, 21target port limits and specifications, 26

LLUNs

discovering on host, 95exporting to host, 92marked as offline after an upgrade, 8

Mmultipathing

configuring ESX/ESXi, 42configuring round robin, 40FC, 38FCoE, 38iSCSI, 38

Oout-of-space condition

for ESX 4.1, ESXi 5.x, or ESXi 6.0, 54

PPE, 103PE LUN, 13, 24

Protocol Endpoint LUN, 13Peer Persistence

configuring, 131Persistent Checksum over FC, 20Persistent Ports

connectivity guidelines for FC, 20for FC, 19setting up for FC, 20

ports3PAR Persistent Ports for FC, 193PAR Persistent Ports for FCoE, 34configure, 9configuring for an FCoE host connection, 61FC target port limits, 18FC target port specifications, 18FCoE target port limits and specifications, 33iSCSI target port limits and specifications, 26Persistent Ports for iSCSI, 27setting up for an iSCSI connection, 21setting up for FC, 10setting up iSCSI persistent port, 27

140 Index

primitivessummary, 56TP LUN Reporting, 56VAAI, 51

protocol endpoint see PEProtocol Endpoint LUN see PE LUN

Rremote support, 138restrictions

FCoE target port limits and specifications, 33iSCSI target port limits and specifications, 26

SSCSI commands support

primitives, 51, 56using ESX/ESXi 4.1 plug-in, 51

Smart SANFC, 16

Software iSCSIconfiguring Software iSCSI initiator, 76installing, 65

specificationsFCoE target port limits and specifications, 33iSCSI target port limits and specifications, 26

SPOCKStorage Single Point of Connectivity Knowledge, 7

storageallocating for access by the ESX host, 89creating, 89creating TDVV, 89creating VVs, 90

storage I/O control, 51support

Hardware iSCSI , 83Hewlett Packard Enterprise, 137

TTDVV

creating, 89thinly deduplicated virtual volumes, 89

thinly deduplicated virtual volumes see TDVVthinly provisioned virtual volume see TPVVTPVV

creating, 90

UUNMAP storage primitive for space reclaim support

using ESXi 5.0 Update 1 with default VMware T10VAAI plug-in and ESXi 6.0, 53

updatesaccessing, 137

upgrading, 8considerations, 8

utilitiesBCU, 9

VVAAI, 51

virtualLUN, 92see also VLUN

Virtual Connect Direct-Attach Fibre Channelstorage, 9

virtual disk VVols see VMDK VVolsvirtual machines see VMsvirtual volumes see VVsVLUNs

creating for export, 93virtual LUNs, 92

VM STUNsee out-of-space condition, 54

VMkernel portcreating for Software iSCSI ESX 4.0, 68

VMsinstalling, 36installing guest OS, 87

VMware virtual volumes see VVolsvolumes

removing, 96vStorage APIs for Array Integration see VAAI

using 3PAR VAAI Plug-in 1.1.1 for ESX/ESXi 4.1, 52using 3PAR VAAI Plug-in 2.2.0 for ESXi 5.x and ESXi

6.0, 52verifying 3PAR VAAI Plug-in 2.2.0, 56

VVolsconfiguration VVols, 99default CPGs, 110, 125defining CPGs, 108installing VASA Provider, 102licenses, 101live-memory snapshot VVols, 99PE and 3PAR, 103protocol endpoint, 103set up with 3PAR OS 3.2.1, 105set up with 3PAR OS 3.2.2, 116swap VVols, 99using, 99VASA Provider PE, 103virtual disk VVols (VMDK VVols), 99

VVs, 89creating, 89, 90exporting, 89fully provisioned, 90thinly deduplicated, 89thinly provisioned, 90virtual volume, 90

Wwebsites, 138

customer self repair, 138

141

to see "hpe 3par vmware esx/esxi implementation guide"

Documents