virtualization planning and implementation guide | experion ......redundantl2.5network layer better...

Experion PKSRelease 516

Virtualization Planning andImplementation Guide

EPDOC-X147-en-D

August 2020

DISCLAIMERThis document contains Honeywell proprietary information.Information contained herein is to be used solely for the purposesubmitted, and no part of this document or its contents shall bereproduced, published, or disclosed to a third party without theexpress permission of Honeywell International Sàrl.

While this information is presented in good faith and believed to beaccurate, Honeywell disclaims the implied warranties ofmerchantability and fitness for a purpose and makes no expresswarranties except as may be stated in its written agreement with andfor its customer.

In no event is Honeywell liable to anyone for any direct, special, orconsequential damages. The information and specifications in thisdocument are subject to change without notice.

Copyright 2020 - Honeywell International Sàrl

2

3

CONTENTSContents 3

Chapter 1 - About this guide 9

Chapter 2 - Getting started with PlantCruise virtualization 13Task overview for preparing a virtualization environment 15

Chapter 3 - Planning for a DCS architecture 19Example DCS architecture virtualization scenario 20

DCS architecture options in a virtualized environment 22

Example DCS architecture in a virtualized environment with non-redundant management network 25

Example DCS architecture in a virtualized environment withredundant management network 27

Example DCS architecture in a virtualized environment withseparate management network 31

Overview of distributing workload across ESXi hosts for a DCSarchitecture 35

Chapter 4 - Planning for a SCADA architecture 39Example SCADA architecture virtualization scenario 40

Example SCADA architecture in a virtualized environment 41

Overview of distributing workload across ESXi hosts for a SCADAarchitecture 42

Chapter 5 - Planning for off-process usage 43

Chapter 6 - Planning the virtualization environment 45Determining the number of ESXi hosts and hardware requirements 46

Host CPU requirements 50

Host memory requirements 52

Storage requirements 53

Contents

Network requirements for a DCS architecture 55

Network requirements for a SCADA architecture 66

Identifying virtualization hardware and software requirements 71

Software requirements 71

Thin client requirements 72

Other requirements 73

Planning for the management ESXi host and management network 74

Planning the management network in a DCS architecture 79

Planning the management network in a SCADA architecture 81

Planning how to organize the vCenter Server 83

VMware inventory objects 83

Datacenter organization guidelines 84

Planning for time synchronization 85

Planning to maintain the VMware environment 87

About vSphere Update Manager 87

Considerations for VMware maintenance when using local storage 87

Considerations about upgrades, patches, and updates 88

Planning security for the VMware environment 90

User accounts, roles, and permissions 90

About host lockdown mode 92

Security hardening 92

Planning to backup the virtual infrastructure and virtual machines 93

Planning for disaster recovery 96

Planning to replicate the virtual machines 97

Chapter 7 - Implementing networks for a DCS architecture 99Implementing a non-redundant management network for a DCSarchitecture 100

Implementing a redundant management network for a DCSarchitecture 108

4

Contents

5

Implementing an optional network configuration where additionalL2.5 management ports are required 116

Implementing a separate management network for a DCSarchitecture 119

Chapter 8 - Implementing networks for a SCADA architecture 131Preparing a gateway router for a SCADA architecture 132

Preparing the production network for a SCADA architecture 133

Configuring the virtual production network for a SCADAarchitecture 134

Implementing the management network for a SCADA architecture 135

Chapter 9 - Preparing an ESXi host 137Configuring the ESXi host local disk array 138

Configuring the ESXi host BIOS settings 139

Configuring your browser 140

Handling certificate error warnings 140

Using Internet Explorer as your browser 141

Using Google Chrome as your browser 143

Installing the ESXi software on a host 143

Configuring the ESXi host network settings 145

Network configuration for a host in a DCS architecture 146

Network configuration for a host in a SCADA architecture 152

Configuring NIC teaming 153

Configuring the virtual switch port security settings 154

Hardening the ESXi host network 155

Configuring ESXi host time synchronization 156

Adding and renaming datastores 156

Configuring the virtual machine load order on the ESXi host 158

Domain name resolution 159

Contents

Chapter 10 - Preparing a vCenter Server 161Installing vCenter Server Appliance 6.5 162

Migrating Windows vCenter to vCenter Server Appliance 6.5 167

Organizing VMware inventory objects 170

Adding the ESXi host to the vCenter Server 171

Configuring VMware sequential MAC address allocation 171

Configuring security in a vCenter Server 172

Creating vCenter roles and assigning privileges 172

Assigning vCenter roles to PlantCruise users or global groups 172

Configuring host lockdown mode 173

Creating and managing virtual machines 175

Chapter 11 - Administering the virtualization environment 177Replicating a virtual machine using vSphere Replication 178

Preparing the vCenter Server for virtual machine replication usingvSphere Replication 178

Preparing the virtual machine for replication using vSphereReplication 179

Restoring and recovering a replicated virtual machine 181

Patching or upgrading the VMware environment 185

Downloading the patches or upgrades 188

Importing the patches or upgrades into Update Manager 189

About baseline and compliance views 189

Remediating patches and upgrades 191

Remediating target objects using Update Manager 193

Remediating target objects manually 195

Upgrading Virtual Hardware and VMware tools versions 200

Upgrading virtual infrastructure software 202

Maintaining hardware devices in a virtualization environment 204

Maintaining the level 2.5 router 204

6

Contents

7

Monitoring the virtualization environment 206

About resource usage 206

About system status 211

Monitoring the virtualization hardware using Dell OpenManage ServerAdministrator 212

Monitoring the virtualization infrastructure from Experion Station 213

Appendix A - Implementing Physical shared storage networks 217About shared storage 218

Redundancy for physical SANs 218

Distributing datastores and volumes across storage networks 218

Preparing the storage network 220

Configuring the storage area network 221

Network configuration for hosts with shared storage 222

Notices 225

Contents

8

Contents

CHAPTER

1 ABOUT THIS GUIDE

This guide provides high-level guidance on how to implement avirtualized PlantCruise environment.

Revision history

Revision Date Description

A August 2020 Initial release of document.

Intended audience

This guide is for people who are responsible for planning, installing,and configuring VMware and PlantCruise components in a virtualizedPlantCruise environment using the Experion Virtualization EssentialsPlatform.

To plan and implement a deployment with the PlantCruise PremiumPlatform, use the Virtualization with Premium Platform Guide as yourstarting point. Refer to the Virtualization Planning and ImplementationGuide as required for tasks such as creating virtual machines andconfiguring vCenter.

Prerequisite skills

It is assumed that you have familiarity with the Microsoft Windowsoperating systems that are supported for the relevant PlantCruiserelease. You should have some experience with, or have completedintroductory training for, the VMware vSphere suite of applications, inparticular ESXi and vCenter. You should also have some experiencewith PlantCruise node installations.

If you are unfamiliar with virtualization technologies, Honeywellrecommends that you read the Honeywell whitepaper, Virtualization

9

Reduces the Cost of Supporting Open Industrial Control Systems. Thiswhitepaper is available from the Honeywell Process Solutions website(http://www.honeywellprocess.com/):

1. Login to the Honeywell Process Solutions websitehttp://www.honeywellprocess.com/.

2. In the Resources column at the bottom of the page, clickWhitepapers.

3. In the Enter search terms box, type virtualization.4. Click Search. The Whitepaper Search Results appear.5. Click the Virtualization Reduces the Cost of Supporting Open

Industrial Control Systems link.

How to use this guide

This guide provides high-level guidance on:

n How to plan for a VMware-based virtualization environment.n How to install and configure VMware virtualization software.n How to create PlantCruise virtual machines in the virtualization

environment.n How to perform system administration of the virtualization

environment.n How to monitor the virtualization environment.

NOTE: Throughout this guide the vSphere Web Client isreferenced. Honeywell recommends using the Flash-based clientfor all vCenter tasks.

Related Honeywell documentation

Document Description

HPS VirtualizationSpecification

This specification identifies the supported PlantCruise andVMware components for PlantCruise virtualization.

Experion Fault TolerantEthernet Specification

This specification identifies the supported FTE switchhardware.

Software InstallationUser's Guide

This guide describes how to install PlantCruise software.You will need to reference this guide to install PlantCruisesoftware on virtual machines.

10

Chapter 1 - About this guide

http://www.honeywellprocess.com/


11

Document Description

SupplementaryInstallation Tasks Guide

This guide describes how to change computer name andsetting up time synchronization.

Experion Network BestPractices

Document ID: WP-07-02-ENG

Wyse 5070 Thin ClientPlanning Installationand Service Guide

This document describes how to configure a thin client toprovide engineering and operational access to VirtualMachines.

You can download Honeywell documentation from thehttp://www.honeywellprocess.com/support website.

VMware vSphere documentation

Download VMware vSphere documentation fromhttp://www.vmware.com/support/pubs/ on the VMware website.

ATTENTION: You should use document versions that match theversion of the products or software that you use. VMwaredocumentation references within this guide are for the vSphere6.5 documentation set.

Other required documentation

Documents for the Dell Essentials Virtualization hardware is availablefrom www.support.dell.com and then enter the Service Tag for thespecific server.

ATTENTION: Use document versions that match the version ofthe products or software that you use.


http://www.honeywellprocess.com/support

http://www.vmware.com/support/pubs/

http://www.support.dell.com/

12


CHAPTER

2 GETTING STARTED WITH PLANTCRUISEVIRTUALIZATION

This guide assumes that you are experienced with PlantCruiseinstallations and virtualization technology and terminology.

Virtualization of a PlantCruise system involves allocating PlantCruisesystem and application nodes to virtual machines rather than physicalmachines. A key aspect of a PlantCruise system that is deployed withvirtualization is the addition of the virtual infrastructure. The virtualinfrastructure is comprised of the hardware and softwarecomponents required to host and manage the system. Someimportant characteristics of a virtualized PlantCruise system include:

n The PlantCruise production workload is consolidated on ESXiserver grade hosts and connected to a production network.

n The production ESXi hosts are also connected to a separatemanagement network to isolate the traffic generated frommanagement of the ESXi hosts and virtual machines from that ofthe production network.

n The production ESXi hosts are optionally connected to a separatestorage network, a hyper-converged vSAN, (where the datastoresare co-located on the ESXi hosts themselves).

n The virtual infrastructure management workload canbe consolidated on a separate ESXi host that can be sharedbetween production and application levels of the system.

n Workload distribution strategies for optimum performance andminimal scope of loss.

n Use of thin clients to provide user interaction with virtual desktops.No PlantCruise application software is installed on the thin client.

n Workload backup and recovery strategies to minimize downtimein the event of a host or virtual machine failure.

n For users of the Premium Platform, hardware and software updateand availability strategies, (DRS, HA, and FT) should be consideredto maximize efficiency and minimize downtime and/or scope ofloss. For more information see Virtualization with PremiumPlatform.

13

If you are familiar with virtualization and VMware

Learn about Description

What PlantCruise nodes can bevirtualized

See the HPS Virtualization Specification.

Supported PlantCruisetopologies

For more information about the supported DCSarchitecture networks, see "Planning for a DCSarchitecture" on page 19.

For more information about the supported SCADAarchitecture networks, see "Planning for a SCADAarchitecture" on page 39.

Hardware and softwarerequirements to supportPlantCruise virtual machines

See "Planning the virtualization environment" onpage 45.

Creating PlantCruise virtualmachines

See the Software Installation User’s Guide.

If you are new to virtualization

Learn about Description

Virtualization andindustrial control systems

See the Honeywell white paper Virtualization Reducesthe Cost of Supporting Open Industrial Control Systems.

Virtualization terms See VMware vSphere Basics Guide from VMware, Inc.

Practical skillsimplementingvirtualization

See VMware vSphere Basics Guide from VMware, Inc.

Virtual network concepts See VMware vSphere Basics Guide from VMware, Inc.

Implementing avirtualized PlantCruiseenvironment

Once you have a practical understanding ofvirtualization, continue with the content presented inthis guide.

14

Chapter 2 - Getting started with PlantCruise virtualization

15

Task overview for preparing a virtualizationenvironment

The following table captures a high level overview of the tasks thatneed to be completed to prepare the virtual environment. For moreinformation about each task, see the relevant chapter within thisguide.

Task Description

Determinearchitecture

Depending on the characteristics of your target system, you canfollow one of two architecture paths:

l Follow the DCS architecture path if your target system has:l A plant network with at least 3 network layers or levelsl An FTE network at level 1 and 2l Multiple clusters and multiple FTE communitiesl A requirement of a high level of security and reliability where

the emphasis is on isolating critical areas of function such aslocal peer-peer control, peer to external peer.

For more information, see "Planning for a DCS architecture" onpage 19

l Follow the SCADA architecture path if your target system has:l A plant network that is inherently flat (Level 1 and Level 2

only with no FTE)l A small number of PlantCruise clusters

TIP: For small PlantCruise systems with FTE (a single FTEcommunity), consider the SCADA architecture path toestablish the management network.

For more information, see "Planning for a SCADA architecture" onpage 39.

Plan forphysicalenvironment

Refer to "Physical and environmental considerations" in theNetwork and Security Planning Guide.

Plan forvirtualizationplatform

Depending on performance and availability requirements, you cancreate your virtual infrastructure using either the Essentials or thePremium platform.


Task Description

For assistance creating your virtual infrastructure with theEssentials platform, see "Planning the virtualization environment"on page 45

For information about creating a virtual infrastructure with thePremium platform, see Virtualization with Premium Platform.

Build thephysicalnetworks

Build the:

l Management network.

See "Implementing networks for a DCS architecture" onpage 99 or"Implementing networks for a SCADA architecture"on page 131.

l Production network

See the Fault Tolerant Ethernet Overview and ImplementationGuide for creating the Level 1 and Level 2 networks,or"Preparing the production network for a SCADA architecture"on page 133.

Install thegateway router(for SCADAarchitecture)

l Configure to route domain controller information between themanagement and production networks.

l Configure to route information to the IT environment/WAN.l Connect the management network.l Connect the production network.l If applicable, connect the storage network.

See "Preparing a gateway router for a SCADA architecture" onpage 132.

Install the ESXihost

l Install the ESXi host hardware.l Connect the ESXi host to all the required networks.l Configure the default virtual connection for the management

network.

See "Preparing an ESXi host" on page 137.

Install themanagementclient

Prepare your internet browser to use for configuration on aseparate computer.

See "Preparing an ESXi host" on page 137.

Configure the Configure the ESXi host network settings.

16


17

Task Description

managementESXi host See "Preparing an ESXi host" on page 137.

Create themanagementworkloads

l Prepare for vCenter Server.l Create management workloads such as Anti-Virus and Backupand Restore appliances. See the Experion Backup and RestoreGuide and the relevant Anti-Virus documentation.

l Install the vCenter Standalone Appliance (VCSA) software.l Add the management ESXi host to the vCenter Server.l Configure vCenter Server security.l Enable the VMware Sequential MAC Address allocation on the

vCenter server.

See "Preparing a vCenter Server" on page 161.

Prepare for virtual machine replication (optional).

See "Planning to replicate the virtual machines" on page 97.

Install theproductionESXi hosts

TIP: Onlyapplicable to theEssentialsplatform

l Install the ESXi host hardware.l Connect the ESXi host to all the required networks.l Configure the default virtual connection for the management

network.l Add the ESXi host to the vCenter Server.

See "Preparing a production ESXi host" on page 1.

If using shared storage, ensure that the ESXi host can connect tothe shared storage.

See "Implementing Physical shared storage networks" onpage 217.

Create a virtualmanagementclient

l This virtual machine can be used in preference to the physicalmanagement client and contains the same software as thephysical management client.

l The physical management client should continue to bemaintained.

Secure thel Ensure the appropriate security groups and policy is in place


Task Description

system on the vCenter Server.l Lockdown the ESXi hosts.

Install andconfigure theproductionnodes

Create and install the PlantCruise virtual machines and otherproduction workloads.

See the Software Installation User’s Guide.

Install andconfigure thinclients

Plan, install and configure thin clients.

See the Wyse Z90 Thin Client Planning Installation and ServiceGuide.

Configuredisasterrecoverystrategy

l Install and configure NAS devices. For more information, seethe Backup and Restore Guide.

l Configure Experion Backup and Restore (EBR). For moreinformation, see the Backup and Restore Guide.

l Configure virtual machine replication (optional). For moreinformation, see "Planning to replicate the virtual machines" onpage 97

l Protect workloads using FT, DRS, and HA. For moreinformation, see the Virtualization with Premium Platform guide.

Upgradevirtualizationhardware andtools

Upgrade virtual hardware and VMware tools versions.

See "Patching or upgrading the VMware environment" onpage 185, and the Software Installation User’s Guide.

18


CHAPTER

3 PLANNING FOR A DCS ARCHITECTURE

In this section:

Example DCS architecture virtualization scenario 20

DCS architecture options in a virtualized environment 22

Example DCS architecture in a virtualized environment with non-redundant management network 25

Example DCS architecture in a virtualized environment withredundant management network 27

Example DCS architecture in a virtualized environment withseparate management network 31

Overview of distributing workload across ESXi hosts for a DCSarchitecture 35

19

Example DCS architecture virtualizationscenario

Discussion within this guide about supported DCS architecturetopologies is based on the virtualization of the following examplescenario.

Figure 3-1: DCS architecture virtualization scenario

In this example scenario, there are two PlantCruise clusters to bevirtualized at Level 2:

n PlantCruise cluster 1

This PlantCruise cluster includes redundant PlantCruise servers,

20

Chapter 3 - Planning for a DCS architecture

21

multiple ACEs, multiple Flex stations, multiple Console Stations,redundant OPC Server, redundant MAC and MAU servers, and anFDM.

n PlantCruise cluster 2

This PlantCruise cluster includes redundant PlantCruise servers,multiple ACEs, multiple Flex stations, multiple Console Stations, aSIM-ACE, and a SIM-C300.

Each PlantCruise cluster hosts a Windows domain controller.



22

DCS architecture options in a virtualized environmentA virtualized DCS architecture requires ESXi hosts at Level 2 and Level 3 to support the production workloads as wellas at least one ESXi host to support the management workload. While the network connectivity of the virtualizedproduction workloads is the same as in a physical system (for example, FTE at Level 2 with network isolation fromLevel 3), there is an additional network introduced for the management infrastructure. This management networkconnects all ESXi hosts in order to provide management access to all elements of the virtualized infrastructure.

Note that this Level 2.5 network layer is introduced to both physical and virtual systems to expand high securitycommunications beyond the FTE community. The bare metal system requires this for cross community peer to peerand PCDI communications. DCS architecture with virtualization is leveraging this same network approach to providea secure method of distributing the management between FTE communities without open access of these networksto Level 3.

The table below identifies three management network deployment options for local storage deployments at Level 2with recommended usages.

ProductionNetworkSecurity

MgmtNetworkHighAvailability

SupportsinterclusterCommsBestPractice

RelativeCost

Usage Go To

Non-redundantmanagement networkwith single routedconnection to L2.5network layer

Good No No Low small-medium systemdeployment

single PlantCruise cluster

multiple PlantCruise clusterswith no requirement forintercluster comms

"Example DCSarchitecture in avirtualizedenvironment with non-redundantmanagement network"on page 25


23




RelativeCost

Usage Go To

local storage solution

new sites

existing sites with lownumber of PlantCruiseclusters

single mgmt network forPCN levels (L2, L2.5, L3, andL3.5)

Redundantmanagement networkwith dual–routedconnection to L2.5network layer

Good Yes Yes Medium New sites with:

multiple PlantCruise clustersrequiring interclustercomms

shared storage at L3

"Example DCSarchitecture in avirtualizedenvironment withredundantmanagement network"on page 27

Non-redundantmanagement networkwith single switchconnection toredundant L2.5 networklayer

Better No No High Existing sites where thereare a large number of FTEcommunities and / or L2.5port usage is limited and/oruse of VLANs is not possible

"Example DCSarchitecture in avirtualizedenvironment withseparate managementnetwork" on page 31


24




RelativeCost

Usage Go To

single PlantCruise cluster

multiple PlantCruise clustersrequiring interclustercommunications


separation of managementLANs between security levelsto support ACLs or firewalls

Management networkwith switch connectionto redundant L3network layer

Better No No High Existing sites where thereare a large number of FTEcommunities and / or L3port usage is limited and/oruse of VLANs is not possible

single or multiplePlantCruise clusters


separation of managementLANs between security levelsto support ACLs or firewalls

"Example DCSarchitecture in avirtualizedenvironment withseparate managementnetwork" on page 31

25

Example DCS architecture in a virtualizedenvironment with non-redundant managementnetwork

Use of a single dedicated switch/router is recommended for single orlow cluster DCS deployments with virtualization. This exampleillustrates the option to share the management infrastructure at L2with Level 3 and Level 3.5. If site security best practices restrict accessat Level 3 and above, then a separate management infrastructurewould need to reside at any level requiring isolation.

Consider the following connectivity and configuration strategies listedbelow to deploy L2.5 with a switch/router for DCS virtualization:

n Each FTE community is connected to L2.5 as routed portn Non-redundant management network is implemented as vLAN in

L2.5 routerl Recommendation is to plan/design for dual L2.5 routers but

physically connect with single L2.5l Consider the option to implement dual connection from host to

the single router. This option is shown in the figure below with adashed line

n ACLs are defined at L2.5 to limit access to management networkn Management infrastructure is common between L2, L3, and L3.5

l Separate management subnet defined at L3 and routedthrough L3 routers to enable secure access for sharedmanagement infrastructure

l Separate management subnet defined at L3.5 and routedthrough L3 routers to enable secure access for sharedmanagement infrastructure

n ACLs defined at L3 and L3.5 to limit access to managementnetwork and to enable peer to peer DC synchronization


Figure 3-2: Example DCS architecture network topology with non-redundant management network

26


27

Example DCS architecture in a virtualizedenvironment with redundant managementnetwork

Use of redundant dedicated switch/routers is the recommendationfor multi-cluster DCS deployments with virtualization. This exampleillustrates the option to share the management infrastructure at L2with Level 3. If site security best practices restrict access at Level 3and above, then a separate management infrastructure would need toreside at any level requiring isolation.

This example illustrates a separate management infrastructure atLevel 3.5. If site security best practices do not restrict access at Level3, then consider the option to share the management infrastructure atLevel 3 and Level 3.5. For an example illustration of this option, seethe diagram in “Example DCS architecture in a virtualizedenvironment with non-redundant management network”.

Consider the following connectivity and configuration strategies listedbelow to deploy L2.5 with redundant switch/routers for DCSvirtualization:

n Each FTE community is connected to L2.5 as routed portn Management network is implemented as vLAN in L2.5 Routers

l Crossover is established between A and B routersl Router redundancy is defined (such as HSRP)

n ACLs are defined at L2.5 to limit access to management networkn Management infrastructure is common between L2 and L3

l Separate management subnet defined at L3 and routedthrough L3 routers to enable secure access for sharedmanagement infrastructure

l ACLs defined at L3 to limit access to management network andto enable peer to peer DC synchronization

n Separate management network with infrastructure at L3.5


Figure 3-3: Example DCS architecture network topology withredundant management network

Option for shared storage at level 3

There is also an additional network at Level 3 for the optional use ofshared storage. In a Level 3 shared storage solution, the SAN(s) wouldhost the Level 3 production workload. Storage traffic must beseparated from production traffic. The figure above shows one optionto separate the storage traffic from the Level 3 production traffic byrouting the traffic through the Level 3 routers. The use of this optionwould require an additional vLAN to be defined in the Level 3 routers.

28


29

An alternative approach to separate the storage traffic is to connectthe redundant SANs to a redundant pair of storage network switches.This would be similar to the shared storage topology shown for usewith SCADA architecture. The use of separate network switches shouldbe strongly considered if the Level 3 workload is disk I/O intensive.

For either approach, the solution requires aggregated ports betweenthe redundant pair of switch-routers. For more information aboutplanning and deploying shared storage, see the related topics.

The figure above shows the Level 2 production ESXi hosts and theLevel 2 thin clients independent of PlantCruise clusters. For moreinformation about the deployment considerations for multiplePlantCruise clusters at Level 2, see the related topics.

Optional network configuration where additional L2.5management ports are required

Depending on your system architecture you may need additionalManagement Network ports over those provided by your L2.5 routers.In this instance you can connect additional 1GB switches tomanagement ports on the L2.5 Router A and B configured tocommunicate on the existing management network.


Figure 3-4: Example DCS architecture network topology withredundant additional Management Network switches

30


31

Example DCS architecture in a virtualizedenvironment with separate managementnetwork

This example uses either the same redundant L2.5 backbone that waspreviously introduced, or an existing redundant L3 backbone (i.e. noL2.5). The option with L2.5 provides both high security and highavailability for communication between FTE communities. The optionwithout L2.5 is for those sites that do not require the communicationbetween FTE communities. In either option, the management networkis implemented as a separate network with its own managementswitch that is connected to L2.5 or L3 via a routed port. This approachis more suited for brown field sites where existing networkinfrastructure limits the number of available ports that can be usedfor the virtual infrastructure.

This can be implemented with a single management network. Anadditional option would be to implement this with redundantmanagement switches. The management infrastructure at L2 or L3 isnot shared. The management infrastructure that resides at Level 3 canbe optionally shared with L3.5.

Consider the following connectivity and configuration strategies listedbelow to deploy L2.5 or L3 with redundant switch/routers for DCSvirtualization:

n Redundant L2.5 and/or L3 with:l Crossover is established between A and B routersl Router redundancy is defined (such as HSRP)

n Each FTE community is connected to L2.5 as routed portn Management Network at L2 is isolated from L2 production

l Management infrastructure (management host, physicalmanagement client and backup device) with managementswitch established on separate subnet

l Consider the following recommendations for the managementswitch:l The same switch device as recommended for the

management network in the SCADA architecturel Any of the current set of recommended FTE switches that

has enough gigabit ports for management infrastructureconnections


l Management network is connected to L2.5 as routed portl Consider the option to implement dual connection from L2

hosts to the single router. This option is shown in the figurebelow with a dashed line.

l ACLs defined at L3 to limit access to management network andto enable peer to peer DC synchronization

n ACLs are defined at L2.5 to limit access to management networkn Separate Management network and infrastructure resides at L3

and is shared with L3.5l Separate management subnet defined at L3.5 and routed

through L3 routers to enable secure access for sharedmanagement infrastructure

l Consider the option to implement dual connection from L3 orL3.5 hosts to the single router. This option is shown in thefigure below with a dashed line

32


33

Figure 3-5: Example DCS architecture network topology with separatemanagement network connected to L2.5


Figure 3-6: Example DCS architecture network topology with separatemanagement network connected to L3

34


35

L2 management infrastructure deployment optionsfor multi-cluster systems

The figure above illustrates a management infrastructure deploymentstrategy for multiple PlantCruise clusters. In this example, thedeployment uses a silo or island approach where each PlantCruisecluster has its own virtual infrastructure management (i.e.management host, physical management client and backup device). Ifa silo approach is not required, then deployment of a singlemanagement infrastructure that is shared by all the clusters at L2 isrecommended.

Overview of distributing workload across ESXihosts for a DCS architecture

NOTE: For information about distributing loads across PremiumPlatform clusters, see the Virtualization with Premium PlatformGuide.

An ESXi host’s workload consists of the set of nodes that reside asguests on that host. When planning for the distribution of the nodes,you need to consider the type of workloads to be deployed.

Process operational workloads refer to the set of nodes that executeat Level 2. Application operational workloads refer to the set of nodesthat execute above Level 2.

Distributing nodes across the ESXi hosts is based on the high leveldeployment model. In a virtualized environment, Level 2.5 isintroduced into this model. It contains the workload used to managethe ESXi hosts and other components that form part of the virtualinfrastructure.

Figure 3-7: Multiple plant level deployment model with a managementnetwork at Level 2.5


The process of distributing nodes begins with the identification andseparation of workloads. Begin by separating the workload by plantlevel. Separate the Level 2 workload by PlantCruise cluster.

Example process operationalworkloads

Example applicationworkloads

Example managementworkloads

l PlantCruise server A -Cluster 1

l PlantCruise server B -Cluster 1

l Flex Stations - Cluster 1l Console Stations - Cluster

1l ACEs - Cluster 1l MAC server X - Cluster 1l MAC server Y - Cluster 1l MAU server X - Cluster 1l MAU server Y - Cluster 1l OPC server A - Cluster 1l OPC server B - Cluster 1l Domain controller -

Cluster 1l FDM gateway- Cluster 1l FDM client- Cluster 1l FDM server- Cluster 1

l eServerl EMDB server Al EMDB server Bl PlantCruise Application

Server (EAS)l Flex Stationsl Domain controllerl Honeywell Advanced

Applications

l vCenter Serverl Management Clientl Backup appliancel Anti-virus Appliancel Domain controller

36


37

Example process operationalworkloads



l PlantCruise server A -Cluster 2

l PlantCruise server B -Cluster 2

l Flex Stations - Cluster 2l Console Stations - Cluster

2l ACEs - Cluster 2l SIM-ACE - Cluster 2l SIM-C300 - Cluster 2l Domain controller -

Cluster 2

ATTENTION: In a virtualization environment, a domaincontroller is required for the management network, which isintroduced as part of the virtualization environment.When using Virtual Machine replication, the task of planningthe workload deployment requires that the original virtualmachine and the cloned replica virtual machine must both beconsidered in the workload deployment. For more information,see the related topics.

For most virtualization environments, it is expected that themanagement workloads can be hosted on a single ESXi host.

Distribution of workload is a significant contributing factor to thenumber of required ESXi hosts as well as the capacity and expectedperformance of each ESXi host. For more information on how toallocate operational workload to ESXi hosts, see the related topics.For more information about the host considerations for themanagement workload, see the related topics.


38


CHAPTER

4 PLANNING FOR A SCADA ARCHITECTURE

In this section:

Example SCADA architecture virtualization scenario 40

Example SCADA architecture in a virtualized environment 41

Overview of distributing workload across ESXi hosts for a SCADAarchitecture 42

39

Example SCADA architecture virtualizationscenario

Note: This section is only applicable to the Essentials Platform.

Prior to implementing Experion virtualization for a SCADAarchitecture, understand the supported topologies.

Discussion within this guide about supported SCADA architecturetopologies is based on the virtualization of the following examplescenario.

Figure 4-1: SCADA architecture virtualization scenario

ATTENTION:l For the virtualization of OPC servers, it is assumed that these

servers use an Ethernet network connection to theprogrammable logic controller (PLC).

l Advanced Applications include AAM, Business FLEX, ProfitSuite, and Procedure Analyst. While this scenario includesAdvanced Applications, this guide does not provide anyinstructions for the installation or deployment of theseapplications in a virtualization environment.

40

Chapter 4 - Planning for a SCADA architecture

41

Example SCADA architecture in a virtualizedenvironment

Figure 4-2: Example SCADA architecture network topology using localstorage on the ESXi hosts

Figure 4-3: Example SCADA architecture network topology usingshared storage


Overview of distributing workload across ESXihosts for a SCADA architecture

An ESXi host’s workload consists of the set of nodes that reside asguests on that host. When planning for the distribution of the nodes,you need to consider the types of workloads to be deployed.

The process of distributing nodes begins with the identification andseparation of workloads.

Example processoperational workloads



l PlantCruise server Al PlantCruise server Bl OPC serverl Flex Stationl Engineering Stationl Secondary domain

controllers

l eServerl Honeywell Advanced

Applications

l vCenter Serverl Management Clientl VMware Data Recovery

(vDR) appliancel Primary domain

controller

ATTENTION: In the example scenario, there is only one domaincontroller. In a virtualization environment, more than onedomain controller is required (a domain controller is requiredfor the management network, which is introduced as part of thevirtualization environment, and another domain controller isrequired for the process operational network). The primarydomain controller, which is connected to the managementnetwork, is considered part of the management workload, whilesecondary domain controllers are considered part of theprocess operational workloads.

For most virtualization environments, it is expected that themanagement workloads can be hosted on a single ESXi host.

Distribution of workload is a significant contributing factor to thenumber of required ESXi hosts as well as the capacity and expectedperformance of each ESXi host. For more information on how toallocate operational workload to ESXi hosts, see the related topics.

42


CHAPTER

5 PLANNING FOR OFF-PROCESS USAGE

If you are implementing a PlantCruise virtual system for off-processusage, you should follow the relevant planning and implementationtopics for DCS and SCADA-based architectures. An off-process systemis not intended for on-process control and is typically isolated froman on-process systems.

Depending on the intended usage, an off-process system, whencompared to an on-process system, may have the following:

n A flatter network topology.Consider the network security, functionality, and performanceofferings of an on-process system before discounting networkconfigurations.

n Higher consolidation ratios.Consider the scope of loss and the acceptable performance ofESXi hosts for off-process usage.

43

44

Chapter 5 - Planning for off-process usage

CHAPTER

6 PLANNING THE VIRTUALIZATIONENVIRONMENT

The following topics describe the planning considerations for aPlantCruise virtualization environment.

In this section:

Determining the number of ESXi hosts and hardwarerequirements 46

Identifying virtualization hardware and software requirements 71

Planning for the management ESXi host and managementnetwork 74

Planning how to organize the vCenter Server 83

Planning for time synchronization 85

Planning to maintain the VMware environment 87

Planning security for the VMware environment 90

Planning to backup the virtual infrastructure and virtualmachines 93

45

Determining the number of ESXi hosts andhardware requirements

Before identifying any hardware requirements, the PlantCruise systemmust be defined. This requires a knowledge of all planned PlantCruisevirtual nodes and the topology that will be used in the system.

Use the following steps to estimate your virtualization hardwarerequirements.

Planning workload grouping

The aim of workload grouping is to form groups of virtual machinesthat will connect to the same network. To do this, identify workloadsthat will operate on the same network level. Workloads belonging tothe Level 2 network (for DCS architectures) or the production network(SCADA architecture), should also be grouped into PlantCruiseclusters.

The groupings that are achieved by this differentiation are now readyfor further workload distribution assessment.

DCS architecture example

An example DCS architecture grouping at this point maybe:

n Level 2 PlantCruise cluster 1

n Level 2 PlantCruise cluster 2

n Level 3 group

n Level 3.5 group

n Management workload (Level 2.5)

SCADA architecture example

An example SCADA architecture grouping at this pointmay be:

46

Chapter 6 - Planning the virtualization environment

47

n Process operational workload

n Application operational workload

n Management workload

Determining host groups

The aim of creating a host group listing is to form the first tentativegroups of virtual nodes that can be consolidated into an ESXi host.Host performance and capacity is not considered at this point.

To create host groups, assess the availability requirements for eachvirtual node in a given workload group and place in a host group. Atthis point in time, it is also recommended to consider scope of losswhen assigning multiple workloads to a given host group. Forexample, assigning all flex stations, console stations, to a given hostgroup should be avoided. Distribute them evenly across applicablehost groups.

ATTENTION:l Redundant applications should not be placed in the same host

group.

l No more than one Windows domain controller can run in thesame host group.

l Replicated VMs or fault tolerant VMs should be included indeployment calculations

DCS architecture example

Level 2 PlantCruise cluster 1 host group A:

n PlantCruise server A

n Flex Station 1

n Flex Station 3

n ACE 1


n ACE 3

n ACE 5

Level 2 PlantCruise cluster 1 host group B:

n PlantCruise server B

n Flex Station 2

n Flex Station 4

n ACE 2

n ACE 4

This example is not complete, as it does not list all nodesfor PlantCruise cluster 1, nor does it show all groupings,such as PlantCruise cluster 2, Level 3 group, Level 3.5groups, and management workload.

SCADA architecture example

Process operational workload group A

n PlantCruise server A

n OPC server

n Flex Station

Process operational workload group B

n PlantCruise server B

n Engineering Station

n Windows domain controller

Application operational workload

n eServer

n Honeywell Advanced Applications

Management workload

48


49

n vCenter Server

n Management Client

n VMware Data Recovery (vDR)

n Windows domain controller

Identifying host group resource requirements

The aim of this step is to calculate the resources for each host group.

To perform this calculation, use the virtual machine workload datacontained in the HPS Virtualization Specification and derive the totalresource requirements for each host group.

Regardless of whether hyper-threading is enabled or disabled, thetotal number of CPUs from VM workloads allocated to a host mustnot exceed twice the number of physical cores.

Defining the host hardware specifications

The aim of this step is to define and calculate the resources that willbe provided by a given hardware platform.

Platform capabilities should be assessed by CPU, hard disk capacityand performance, memory, and network bandwidth. Each of theseparameters need to be considered for workload assignment and mustbe known at this point.

Assessing the targeted host hardware against thehost groups

The aim of this step is to establish if the assessed hardware is capableof providing the resources required by each host group.

To do this, compare the host group resource requirements with thehardware platform capabilities.

If the platform is not able to provide the calculated resources, thefollowing should be considered:

n Adjust the configuration of the platform to provide the requiredresources.


n Adjust host group size by redistributing workloads to additionalhost groups.

For additional information on resource requirements, see the relatedtopics.

When host groups are assessed and resource requirements are met,each host group can now form the planned ESXi host consolidatedworkload.

Host CPU requirementsYou need to consider host CPU requirements.

When considering virtual machine consolidation, the assignment ofvirtual CPU is an important factor as it will directly affect theperformance of the virtual machines.

Every ESXi host has a specified number of logical processors andphysical cores. If hyper-threading is enabled, the number of logicalprocessors must be twice the number of physical cores, otherwise thevalues can be the same. If hyper-threading is available as an option onthe CPU, it should always be enabled in the BIOS.

Some mitigations for Intel side-channel vulnerabilities, such as for theForeshadow/L1TF issue, may result in the ESXi hypervisor internallydisabling hyper-threading. The ESXi hypervisor knows which CPUs areimpacted by this vulnerability and which are not, so when ESXi L1TFmitigation is enabled it will enable or disable hyper-threadingaccordingly.

To ensure VM performance at normal run time and duringmaintenance activities when additional transitory load may berequired, Honeywell recommends that regardless of the use of hyper-threading by the ESXi hypervisor and its impact on the number oflogical processors, the total number of assigned vCPUs on an ESXihost should not exceed twice the number of physical cores.

Example of how to calculate the number of vCPUs thatcan be used on an ESXi host

An ESXi host has an Intel Xeon E5-2680v4 installed. ThisCPU has 14 physical cores and includes hyper-threadingfunctionality, however as it is impacted by the Intel sidechannel vulnerabilities, and this mitigation has been

50


51

enabled on the system, its logical core count shouldmatch the physical cores at 14 as ESXi will disable hyper-threading. Honeywell recommends using up to 28 vCPUsof workload on this host (2 x logical core count of 14).

An ESXi host has an Intel Xeon Gold 6148 installed. ThisCPU has 20 physical cores, includes hyper-threadingfunctionality, and is impacted by the Intel side channelvulnerabilities. However, the system is not currentlyrunning with the side channel vulnerability mitigationenabled, so it is effectively running with 40 logical cores(2 x 20 physical cores). Honeywell recommends using upto 40 vCPUs of workload on this host (2 x logical corecount of 20).

CPU MHz considerations

The clock speed of the hardware platform CPU is important. This is inaddition to the virtual CPU requirements and should be considered toensure that the hardware can satisfy CPU time for the virtualmachines. The performance matrix table in the HPS VirtualizationSpecification includes a CPU MHz value for each PlantCruise node.

The sum of the CPU MHz for all virtual machines in each host groupshould not exceed 80% of the available hardware MHz so as toensure that hypervisor overhead is included in planning. Note that thetotal CPU MHz does not include any extra MHz for hosts utilizinghyper-threading, so overall CPU MHz totals are not impacted by thedisabling of hyper-threading to mitigate the Intel side channelvulnerabilities on affected CPUs.

ATTENTION: If you are planning to use the PremiumVirtualization Platform that leverages VSAN, each host requiresan additional 10% CPU overhead on top of the virtual machineCPU Mhz. This consideration is important as it may limit theability to consolidate as many virtual machines in each ESXiHost.

Examples of how to calculate the available hardware MHz


An ESXi host has an Intel Xeon E5-2680v4 installed, withIntel side-channel mitigation enabled in ESXi (i.e., hyper-threading disabled). This CPU has 14 physical cores at2.4GHz per core. The total available hardware MHz is36400 MHz (2400 x 14 = 36400).

An ESXi host has an Intel Xeon Gold 6148 installed, withIntel side-channel mitigation disabled in ESXi (i.e., hyper-threading enabled). This CPU has 20 physical cores at2.4GHz per core. The total available hardware MHz is48000 MHz (2400 x 20 = 48000). Note the 40 logicalcores have no impact on total MHz availability.

Host memory requirementsFor estimating and planning purposes, the total allocated virtualmemory must be equal to, or less than, the total physical memoryavailable on the ESXi host.

Memory overcommitment

Unused memory (the difference between allocated memory andactive memory) can be shared with other virtual machines. Inaddition, VMware uses other techniques to optimize memory usage.

These memory management techniques, collectively known asmemory overcommitment, enable you to run virtual machines wherethe total allocated memory is greater than the physical memoryavailable on the ESXi host.

On-process usage Off-process usage

Reduced performancemay be unacceptableand the level ofmemoryovercommitment mayneed to be reduced toimprove performanceof the virtual machines.

The total allocatedmemory must be equalto, or less than, thetotal physical memory

It may be acceptable for slightly reduced performance forthe benefit of improved host utilization and increased costsavings.

Extreme use of memory overcommitment can result inpoor system performance. Honeywell recommends thatyou limit the total active memory to no greater than 100%above the physical memory available on the ESXi host. AnESXi host level alarm will trigger if the host has reached aconfigured level of usage of physical memory. The defaultusage level is 90%.

If host memory usage alarms are generated and/or your

52


53

On-process usage Off-process usage

available on the ESXihost.

performance is poor, then you should consider reducingthe amount of memory overcommitment. The amount ofallocated memory for each PlantCruise virtual machineshould not be adjusted. Instead, you need to reallocate thevirtual machines across ESXi hosts to achieve the bestperformance/cost/host utilization characteristics.

ATTENTION: For both on-process and off-processusage, you should monitor the active memory foreach virtual machine and factor into your calculationadequate memory to handle peak memory usageacross the virtual machines.

Storage requirementsHoneywell supports local host storage, shared storage, hyper-converged storage (vSAN), and network attached storage (NAS). Thisguide describes the Essentials Virtualization Platform, and the localstorage that it provides. Use the Virtualization with Premium Platformguide for detailed planning and configuration guidance on VMware'shyper-converged storage solution (vSAN).

ESXi uses virtual disk files as the disk drives for virtual machines. Eachvirtual machine stores these files (with the .vmdk file extension) in adirectory in a datastore. You need to identify:

n The location of the datastores. There are two types of storage thatcan be used for virtual machines:

l Local storagel Shared storage, including SANs and vSANs

n The performance requirements of the disk array. Consider theplanned virtual machine workload for each host with local storageand the total virtual machine workload for shared storage. Use theperformance matrix table in the HPS Virtualization Specification tocalculate the total average IOPs and total maximum IOPs for eachworkload and ensure the disk arrays can meet that demand.Inadequate disk performance affects the performance of all virtualmachines on that host or shared storage.

n The size of the datastore, taking into consideration the number ofvirtual machines, the guest operating systems, the application


software, and additional space for data and growth.The total disk space of the disk/volume must be greater than theallocated disk size requirements for each PlantCruise virtualmachine, plus additional space for swap files and partition, and adiagnostic partition.If you also want to store snapshots, ISO images, virtual machinetemplates, or floppy disk images, you will need additional diskspace.

ATTENTION: Honeywell does not recommend the prolongeduse of snapshots on a production virtual machine. If youdetermine that a snapshot must be used the file system needsto be quiesced (via the checkbox option) and the “Snapshot thevirtual machine’s memory” option needs to be disabled.The maximum size of a virtual disk is restricted by the blocksize when the datastore is created. When creating datastores,keep in mind the size of your virtual disks.For more information about block sizes and datastores, see theBlock size limitations of a VMFS datastore (KB1003565)knowledge base article on the VMware web site.

You also need to consider the storage requirements for virtualmachine backups. Honeywell recommends that Experion Backup andRecovery (EBR) be used as the backup solution for virtualizedenvironments.

Local storage

Virtual machines images are stored directly on the disks within theESXi host that is hosting the virtual machine.

During the installation of ESXi, empty disk space on the partitionwhere ESXi is installed is automatically partitioned and formatted, andconverted into a datastore. Disk space on empty partitions are alsoformatted and converted into datastores. Hardware vendor partitions,such as Dell Utility partitions are retained and not formatted.

You should plan a naming convention for the datastores created onlocal storage. When using vCenter Server, you will be presented with alist of datastores, which are easier to manage if they are uniquely andconsistently named.

54


http://kb.vmware.com/kb/1003565

55

Shared storage

Virtual machine images are stored on shared storage devices that oneor more ESXi hosts can access.

Shared storage is a key requirement to enable advanced virtualizationfeatures such as vMotion, High Availability, Fault Tolerance, and SiteRecovery Manager (SRM) for business continuity and disasterrecovery. If none of these features are required now, or in the future,local storage will meet your needs.

You should plan a naming convention for the volumes and datastorescreated on shared storage. The volume name and the datastore namemay differ. When using vCenter Server, you will be presented with alist of datastores, which are easier to manage if they are uniquely andconsistently named.

Backup storage

For backup storage guidance, refer to the Experion Backup andRestore (EBR) documentation.

Network requirements for a DCS architectureThere are network-related requirements that should be considered tooptimize the alignment between the virtual network (vNetwork) andthe physical network of a DCS architecture. Prior to determining thenetwork requirements, understand the relationship between thevirtual and physical networks. For more information about theconcepts of virtual networking, see the following sections in thevSphere Networking guide:

n Introduction to Networkingn Basic Networking with vNetwork Standard Switches

Common network configuration decisions

When implementing a production network for a DCS architecture youneed to consider the following:

Consideration Description

Networkspeeds

Network speeds affect the number of physical network connectionsfrom the ESXi host to the production network. The network speedthat the physical production switch supports is the maximum



bandwidth that the production vSwitch can support between virtualmachines running on separate ESXi hosts to the FTE physicalswitches. This limitation needs to be used in any calculation fordetermining the number of virtual machines that can connect toeach vSwitch. More network adapters may be required to satisfy thetotal virtual network throughput.

Networksecurity

You should eliminate all unwanted network traffic in the productionnetwork. A pair of Level 2.5 routers can connect the FTE productionnetworks, management network, and application network (Level 3).

ESXi hosts

PlantCruise DCS architecture includes ESXi hosts for management,L2 production and L3 production. (An ESXi production host containsprocess operational or application operational workloads.) Thenetwork requirements vary depending on the usage of the ESXi host.Each ESXi host must meet the following general hardwarerequirements:

Supported server-grade hardware

Varying levels of capacity are recommended based on the intendeduse of each ESXi host. For example, an operational ESXi host on theproduction network may require more capacity than the managementESXi host on the management network.

For more information about the server-grade hardware requirementsand recommended capacities, see the HPS Virtualization Specificationfrom the Honeywell Process Solutions website.

ATTENTION: Use the same vendor, family, and generation ofCPUs throughout your virtualization environment. This willensure the portability of virtual machines when using vMotion.

The table below identifies general ESXi host hardware considerationsthat impact the networking options.

56


57

ESXi HostUsageDescription

PhysicalSpecification ofNetwork Capacity

Example Usage

ManagementHost

Includes an on-board dual portNetwork InterfaceCard

Management host

Single VMHost

l Includes an on-board dual portNetwork InterfaceCard

l Includes one dualport externalNetwork InterfaceCard

L2 production host with single workloadconsolidation ratios. For an example, see theConsole Station Host description in the HPSVirtualization Specification available from theHoneywell Process Solutions web site.

Lower PointCounts


l Includes twosingle portexternal NetworkInterface Cards

l Option to replacesingle portNetwork InterfaceCards with twodual port externalNetwork InterfaceCards

l L2 production host with multipleworkloads and lower point count. Forexamples, see the Small, Medium or LargeCluster Host descriptions in the HPSVirtualization Specification available fromthe Honeywell Process Solutions web site

l L3 production host with local storage andlow workload consolidation ratios.

Higher PointCounts


l Includes two dualport externalNetwork Interfacecards

l L2 production host with multipleworkloads and higher point count. Forexamples, see the Performance ClusterHost descriptions in the HPS VirtualizationSpecification available from the HoneywellProcess Solutions web site

l L3 production hosts with local storage andhigher workload consolidation ratios

l L3 production host with shared storage


ESXi HostUsageDescription

PhysicalSpecification ofNetwork Capacity

Example Usage

l Supports theability to replacethe dual portNetwork adaptercards with twoquad portNetwork InterfaceCards

In the VMware environment, the physical network adapters on theESXi host are named vmnic <n> , where n is unique numberidentifying a single port on the network interface card. For productionESXi hosts, the following network interface ports are required. Since,it is possible to deploy with single or multi-port network interfacecards, the table below uses the VMware notation where one vmnicequals a single port on a network adapter card.

ATTENTION:l Onboard NICs can be used for connections to the

management and storage networks.

l External Network Interface cards can be used for productionnetworks including FTE. Intel NICs can optionally be used forconnection to FTE networks. For more information, see theFault Tolerant Ethernet (FTE) Specification available from theHoneywell Process Solutions web site

Implementationtype

Storagetype

Minimumnetworkinterface card(NIC)requirements(including anyon-board NICs)

Additional considerations

Level 2, on-process

Local Each ESXi hostmust have fourvmnics:

l Two vmnics

If supported by ESXi host, add twomore vmnics for FTE mapped acrossseparate Network Adapter Cards. Theadditional FTE vmnics are used tominimize scope of loss or increase

58


59

Implementationtype

Storagetype



forconnectingto each FTEnetwork, thatis, the yellowand green.

l Two vmnicsfor themanagementnetwork.These areteamed.

network performance. See the figure"High Capacity ESXi Production Hostwith single FTE connection".

Level 3, on-process

Local Each ESXi hostmust have threevmnics:

l One vmnicforconnectingto the Level3 productionnetwork.

l Two vmnicsfor themanagementnetwork.Thesevmnics areteamed.

If supported by the ESXi host, add onemore vmnic for L3 productionnetwork. The additional vmnics areused to minimize scope of loss orincrease network performance.

Level 3, on-process(option)

Shared Each ESXi hostmust have fivevmnics:

It is recommended to deploy withhigh capacity ESXi host. Consider useof a quad port adapter card to


Implementationtype

Storagetype



l One vmnicforconnectingto the Level 3productionnetwork.

l Two vmnicsfor themanagementnetwork.These vmnicsare teamed.

l Two vmnicsfor thestoragenetwork.

connect to the production andstorage networks.

This guide provides a number of examples that illustrate the detailedESXi host configuration from a network perspective. The figures"Network Connection for Management Host" and "High Capacity ESXiProduction Host with single FTE connection" introduce the ESXi hostnotation that is used throughout the guide. See the ESXi host notationto ensure that:

n Network ports are identified from bottom right to upper rightstarting with vmnic0. Ports that are grouped together indicate useof a multi port Network Interface Card.

n Each ESXi host deployed with local storage, regardless of usage,uses the on-board dual port Network Interface Card to connect tothe management network. This recommendation applies to bothsingle and dual management network implementations. As aresult, vmnic0 and vmnic1 are always assigned to themanagement network.

n Each ESXi host, regardless of usage, is configured to use VMware

60


61

NIC teaming when configuring the uplink configuration from themanagement vSwitch (vSwitch0) to vmnic0 and vmnic1. For moreinformation, see the "NIC Teaming" section that follows

The figure "Network Connection for Management Host" shows anexample of the required connectivity between the vNetwork and thephysical network at the level of an ESXi host used for management.Note the following regarding the connection from virtual machine tovSwitch0 to physical ports vmnic0 and vmnic1:

n The on-board dual port Network Interface Card is recommendedfor connection to the management network. Thisrecommendation applies when implementing a dual or singlemanagement network.

n The management network requires configuration of a singlevSwitch. vSwitch0 is created as a result of ESXi hypervisorinstallation.

n VMkernel connection to vSwitch0 is notated by ESXi managementconnection to vSwitch0. This enables network connectivitybetween the hypervisor and network.

n Management workload (for example, vCenterServer) is connectedto vSwitch0 from single virtual NIC (vNIC0). This is a virtualmachine port group.

Figure 6-1: Network Connection for Management Host

ESXi Management

vmnic 0

Management VMs vSwitch 0

Management B

Management A

vNIC 0

vmnic 1

Notation to indicate vSwitch is configured with NIC Teaming

The figure "High Capacity ESXi Production Host with single FTEconnection" shows an example of the required connectivity betweenthe vNetwork and the physical network at the level of an ESXi hostused for L2 production. The network connection to the managementnetwork is identical to that shown in the figure above, with theexception that a production host does not include management


workload. Note the following regarding the FTE connection fromvirtual machine to vSwitch1 and vSwitch2 to physical ports vmnic2and vmnic4:

n When creating your VMs for use with FTE, NICs must be createdsequentially so that the MAC addresses are also sequential. If aNIC has to be deleted and re-added, all NICs should be deletedand re-added.

n A dual port Network Interface Card enables a single connection tothe yellow side of the physical FTE network. An additional dualport Network Interface Card enables a single connection to thegreen side of the physical FTE networkl Note the recommendation of two dual port Network Interface

cards for single FTE connection. Separation of FTE betweentwo physical cards minimizes the scope of loss. In this case, thespare vmnics (vmnic3 and vmnic5) are reserved for future use.An example of future usage might be that unacceptablenetwork performance of the PlantCruise workload may warrantconsideration to split the FTE workload between two FTEconnections (see below).

l Low capacity ESXi Production hosts (not shown) use either onedual port Network Interface Card or two single port NetworkInterface cards.

n Single FTE connection requires configuration of two vSwitches –one for yellow and one for green.l Dual FTE connection option (not shown): Configure an

additional yellow vSwitch (vSwitch3) to connect to vmnic3.Configure an additional green vSwitch (vSwitch4) to connect tovmnic5. Consider this option if the host is planned to deployFTE workload that warrants separation

n Each virtual machine with FTE is configured with two virtual NICs– one for yellow and one for green. In the single FTE connectionexample, Yellow virtual NIC is connected to vSwitch1. Green virtualNIC is connected to vSwitch2..l Dual FTE connection option (not shown): virtual machines with

FTE are either connected to vSwitch1 and vSwitch2 OR tovSwitch3 and vSwitch4

Figure 6-2: High Capacity ESXi Production Host with single FTEconnection

62


63

FTE Network Yellow

Management Network A

ESXi Management vmnic 0

vSwitch 0

Virtual Machine with FTE

vmnic 1

Management Network B

vmnic 2

vmnic 3

vNIC 0

vNIC 1

Virtual Machine with no FTEvNIC 0

vSwitch 1

Notation to indicate vSwitch is configured with NIC Teaming

vmnic 4

vmnic 5

vSwitch 2

FTE Network Green

NIC teaming

VMware NIC teaming is the grouping together of several physicalNICs in order to minimize the loss of network connectivity due to aPCI card failure in an ESXi host. VMware NIC teaming also provides aload balancing feature which when used in this context provides for abalance of the number of connections from the virtual network to thephysical network. This is unlike typical load balancing in a physicalnetwork where traffic flow is balanced through all available adapters.to form a single logical NIC.

For on-process systems, VMware NIC teaming is recommended forthe connection to the management network. Each vmnic used in theNIC team for the management network is configured as active inorder to take advantage of the VMware load balancing feature. Thisapplies to traffic from the ESXi host to the physical switches only.There are no required configuration settings on the physical switches.

NIC teaming can also be used at Level 3 for network fault toleranceand improved performance for connection to the storage network.

Below is a summary of the VMware NIC teaming expectations foreach network type.

Network Purpose of NIC teaming

Management For fail over and VMware load balancing.

Storage For fail over and performance.


VMware NIC teaming is a network policy property of each vSwitch.For example, enabling VMware NIC teaming for the managementnetwork connection requires configuration of vSwitch0.

When configuring NIC teaming, ensure that no more than oneadapter is active at any one time. For more information, see"Configuring NIC teaming" on page 153.

Switches and routers

The following table identifies the switch and router requirements,based on each network type.

Network type Switch requirements

FTE network Two switches, one for the green network and another for the yellownetwork. For more information about the FTE switch hardwarerequirements, see the Product Specification for Experion FaultTolerant Ethernet (FTE) from the Honeywell Process Solutions website (http://www.honeywellprocess.com).

Managementnetwork

1. If implementing a non-redundant management network.Use one enterprise-class gigabit switch-router for themanagement network. For systems that implement multiple plantnetwork levels, it is recommended that the management networkbe implemented in between the supervisory control level (Level 2)and the application control level (Level 3). When implemented inthis fashion, the management network becomes Level 2.5 which isconnected to Level 2 and Level 3. Establishing the management(Level 2.5) network also requires the following:

l Allocate four IP addresses in unused subnet for addingredundant IP routing between Level 2.5 and Level 3

l Allocate an unused subnet with enough IP addresses toaccommodate management network connections (forexample, ESXi hosts, NAS, management client).

l Hostname and IP for Level 2.5 router

2. If implementing a redundant management network.Use two enterprise-class gigabit switch-routers for themanagement network. A cross-over cable is required between thetwo switch-routers. For systems that implement multiple plantnetwork levels, it is recommended that the management networkbe implemented in between the supervisory control level (Level 2)and the application control level (Level 3). When implemented in

64



65


this fashion, the management network becomes Level 2.5 withLevel 2.5 Router A and Level 2.5 Router B which are connected toLevel 2 and Level 3. Establishing the management (Level 2.5)network also requires the following:


l Allocate unused subnet with enough IP addresses toaccommodate management network connections (forexample, ESXi hosts, NAS, management client).

l Hostname and IP for Level 2.5 router A

l Hostname and IP for Level 2.5 router B

3. If implementing a separate management network.Use one enterprise-class or small business class gigabit switch forthe management network. For systems that implement multipleplant network levels, it is recommended that the managementswitch network be implemented at Level 2. When implemented inthis fashion, the management network connects to Level 2.5.Establishing the management (Level 2) network also requires thefollowing:


l Allocate an unused subnet with enough IP addresses toaccommodate management network connections (forexample, ESXi hosts, NAS, management client)

l Hostname and IP for Level 2.5 router A

l Hostname and IP for Level 2.5 router B

l Single gigabit port in Level 2.5 router A for connection frommanagement switch

For more information about the recommended switch-routerspecifications, see the HPS Virtualization Specification from theHoneywell Process Solutions web site.


ESXi host IP addresses

The following table identifies the IP address requirements for eachESXi host, based on each network type.

Network type IP address requirements

Managementnetwork

One (1) static IP address for each ESXi Host that is connected tothe management network.

Network requirements for a SCADA architectureThere are specific network-related requirements for a SCADAarchitecture.

Common network configuration decisions

When implementing a production network for a SCADA architecture,you need to consider the following.


Networkavailability.

At least 2 physical network adapters should be usedin each ESXi host for the production network.

For single Ethernet production networks, thenetwork adapters are teamed together for networkavailability. Each teamed NIC connects a vSwitch tothe physical network and switch.

ATTENTION: Ensure that any physicalnetwork adapters used in a NIC team do notbelong to the same embedded or PCI-enetwork card.

For dual Ethernet production networks, singlephysical network adapters connect separatevSwitches to the physical network and switch. Asthere are two paths of communication between thevirtual machines and the SCADA controllers, asingle network adapter failure should not result in aloss of view.

For more information about dual networks, see the“Network redundancy with dual networks” topic in

66


67


the Station Configuration Guide.

Networkspeeds.

Network speeds affect the number of physicalnetwork connections from the ESXi host to theproduction network.

Both 10/100Mbps and 1Gbps networks aresupported for non-FTE production networks. Thenetwork speed that the physical production switchsupports is the maximum bandwidth that theproduction vSwitch can support between virtualmachines running on separate ESXi hosts to theSCADA controllers. This limitation needs to be usedin any calculation for determining the number ofvirtual machines that can connect to each vSwitch.

More network adapters may be required to satisfythe total virtual network throughput.

ATTENTION: The recommended settingsused for NIC teaming do not increase thenetwork bandwidth for single virtualmachines beyond the single physical uplinkspeed to the physical switch. The actualnetwork adapter used is based on the virtualport that the virtual machine traffic enteredthe vSwitch.

Networksecurity.

You should eliminate all unwanted network trafficin the production network. A gateway router canconnect the production networks, managementnetwork, and IT environment/WAN networks.

ESXi hosts

Each ESXi host must meet the following hardware requirements:

n Supported server-grade hardware. For more information aboutthe server-grade hardware requirements, see the HPSVirtualization Specification from the Honeywell Process Solutionsweb site.


ATTENTION: Use the same vendor, family, and generation ofCPUs throughout your virtualization environment. This willensure the portability of virtual machines when usingvMotion.

n For operational ESXi hosts (that is, for ESXi hosts that containprocess operational or application operational workloads), thefollowing network interface cards (NICs) are required.

Implementationtype

Storagetype

Minimum network interface card(NIC) requirements (including anyon-board NICs)

On_process Local Each ESXi host must have fournetwork interface cards (NICs):

l two NICs for the productionnetwork

l two NICs for the managementnetwork

On_process Shared Each ESXi host must have sixnetwork interface cards (NICs):

l two NICs for the productionnetwork

l two NICs for the managementnetwork

l two NICs for the storage network

ATTENTION: Honeywellrecommends the use of NICteaming, which is where twoor more physical adaptersare used to share the trafficload or provide passivefailover in the event of aphysical adapter hardwarefailure.

Off_process Local Each ESXi host must have two

68


69

Implementationtype

Storagetype

Minimum network interface card(NIC) requirements (including anyon-board NICs)

network interface cards (NICs):

l one NIC for connecting to theproduction network

l one NIC for connecting to themanagement network

Off_process Shared Each ESXi host must have fournetwork interface cards (NICs):

l one NIC for the productionnetwork

l one NIC for the managementnetwork

l two NICs for the storage network

ATTENTION: The network interface card (NIC) requirementsdescribed above represent the minimum requirements. Youcan use more NICs for each network.

NIC teaming

For on-process systems, NIC teaming is recommended. NIC teamingis the grouping together of several physical NICs to form a singlelogical NIC. NIC teaming can be used for network fault tolerance andperformance reasons. Here is a summary of the NIC teamingexpectations for each network type.

Network Purpose of NIC teaming

Management For fail over, not performance.

Production For fail over, not performance.

Storage For fail over and performance.


Gateway router

You need a gateway router to enable a connection between theproduction network and the IT environment. The gateway routerprovides access control connections between the production network,management network, IT environment, and optionally, the storagenetwork. If you are creating a virtualization environment from anexisting system, the gateway router may already exist.

Switches

The following table identifies the switch requirements, based on eachnetwork type.


Non-FTEnetwork

A switch for the production network.

Managementnetwork

A switch for the management network, whichmust be a gigabit switch.

ESXi host IP addresses

The following table identifies the IP address requirements for eachESXi host, based on each network type.

Network type IP address requirements

Managementnetwork

One (1) static IP address for the managementnetwork.

Management workloads

The following requirements need to be met for the managementworkloads:

Implementationtype Management workload requirements

On_processusage

The management workloads of vCenter Server,vSphere Update Manager, VMware DataRecovery, and the Windows domain controllermust be virtual machines. Other management

70


71

Implementationtype Management workload requirements

workloads, can be either virtual or physical.However, there must be at least one instance of amanagement client running as a physicalcomputer.

Off_processusage

Can be either a physical server or one or morevirtual machines.

Identifying virtualization hardware and softwarerequirements

You need to consider other software and hardware requirementsprior to implementing a virtualization environment.

Software requirementsSoftware requirements are important and should be defined whileplanning a system.

VMware software and licenses

Ensure that the following VMware software and licensesare considered:

n For every ESXi host, a license based on the number of CPUsockets is required. The latest ESXi hypervisor software image isrequired.

n A vCenter Server license is required.vSAN licenses are required for each host when using the PremiumVirtualization Platform

n If Site Recovery Manager is required, ensure that the BackupControl Center licensing of ESXi hosts and vCenter Server areincluded, plus add the Site Recovery Manager license to both sites.

Operating system software and licenses

Ensure that the operating systems being used in your virtualenvironment are correctly licensed. You can leverage either your ownenterprise operating system agreements or Honeywell can supply


these operating systems for you through WIndows Data Centerlicenses. Additionally, please ensure that there are no special licensingconsiderations for the applications that are running in the virtualizedenvironment.

Other software and licenses

PlantCruise software and licensing should be considered for allrequired nodes in the system. Add licensing as required, based on thePlantCruise system size and additional application and engineeringfunctionality.

Thin client requirementsFor virtualized Experion client nodes, access to the user interface ofthe virtual machine is through a thin client. A thin client is a device thefulfills the traditional computational role of a typical workstationcomputer while the actual workload resides on a remote server, noton the thin client itself.

Thin clients are physically connected to the same network as thevirtual machines that they connect to. Typically, a thin client providesa single LAN connection only. A USB to Ethernet dongle can be usedto provide an additional network port at the thin client to enable FTE.For more information, see the Thin Client Planning Installation andService Guide.

Typically, one thin client is required for each Flex Station or ConsoleStation virtual machine. Depending on your site requirements andusage, thin clients may also be shared for Server, Engineering Station,and other PlantCruise virtual machines.

For more information about the supported thin clients, see the HPSVirtualization Specification.

Refer to the Thin Client Planning Installation and Service Guide forinformation about deploying thin clients.

Consideration Description Options

High availabilityof thin client

Legacy thin clients are not FTE enabled devices.

ATTENTION: If the target virtual machineexists on a different set of FTE switchesthen the thin client, any failure of theupper level switches joining the two

l Consideruse of thinclients witha USB toEthernetadapter toprovide FTE

72


73

Consideration Description Options

networks will not be re routed. capabilitiesat the thinclient.

l For lesscriticalstations,balancedirect thinclientconnectionsacrossyellow andgreennetworkswithoutusing RPP

l ConsiderleveragingHoneywell'snew thinclient, theWyse 5070.Thisproduct hasdual nboardNICs and isqualified forFTE usage.

Other requirementsThere are additional requirements for Experion virtualization.

Windows domain controllers

For on-process usage, an additional Windows domain controller isrequired on the management network. This new domain controller isa peer to any existing domain controllers on the production networkat Level 2 and Level 3. Make the domain controller on themanagement network a PDC emulator master if no other PDC


emulator master exists. The PDC emulator master domain controllermust be configured to synchronize to an external NTP time source.

For off-process usage, a minimum of one Windows domain controlleris required.

Regardless of the usage, the Domain Name System (DNS) serverneeds to be implemented on all domain controllers. Configure DNSwith forward and reverse lookup.

Choosing what nodes to virtualize

You may choose not to virtualize all node types. For moreinformation, including known exceptions, see the HPS VirtualizationSpecification.

Windows operating system updates

You should use existing techniques for applying Windows operatingsystem updates to virtual machines. The PlantCruise best practices forinstalling Honeywell-qualified Microsoft updates are identified inchapter 6, “Microsoft Security Updates and Service Packs” of theNetwork and Security Planning Guide.

Planning for the management ESXi host andmanagement network

One or more management ESXi hosts contain the softwarecomponents for the administration and management of virtualinfrastructure. The management network separates the managementnetwork traffic between the ESXi hosts and the management nodesfrom the process control network (PCN) or production traffic.

Management network

Honeywell recommends that network traffic between ESXi hosts andthe management host be directed to a separate network called themanagement network.

ATTENTION: Separating the management network trafficisolates access to the ESXi hosts, which is in alignment withHoneywell security recommendations.

74


75

One or more of the following items also reside on the managementnetwork:

n Management Client hosted on a physical computer.

n Network Attached Storage (NAS) device(s) will also be placed onthe management network for use by PlantCruise Backup andRestore (EBR) for storage of virtual machine backups of bothmanagement and production workloads. For more information,see the related topics.

Management ESXi host

A management ESXi host runs the management workload.Management workload consists of virtual machines that run on themanagement host and only connect to the management network.Examples of management workload are vCenter Server and EBR. EBRruns as a virtual machine as it is supplied as an appliance. An exampleof a management workload that can run as either a physical machineor a virtual machine is a domain controller.

Management nodes are virtual machines on the management ESXihost. The management ESXi host runs management workloads andhas a connection to the management network only. There is noconnection to the production network. When planning themanagement host consider the number of and type of managementnodes to be included as this will determine the sizing andperformance requirements of the hardware. Consider the role that themanagement ESXi host will play in creation of templates and virtualmachines as this will require more storage capacity to be planned forthis node.

Basic rules used when selecting the management ESXi host hardwareare:

n Ensure that the CPU in the hardware provides as many logicalprocessors as the number of total planned virtual CPU in the host.This consideration includes any template or PlantCruise nodevirtual machines that may be created on the management hosts. Asimple approach for this rule may be selecting a Quad corephysical CPU (with hyper-threading) that supports 8 logicalprocessors. The vCenter requires 2 vCPU, domain controllerrequires 2 vCPU, EBR requires 2 vCPU and this leaves 2 vCPU forstaging PlantCruise virtual machine installations. It is importantnot to underestimate this requirement in the management ESXihost as CPU contention caused by running too many vCPU


compared to logical processors will lead to high CPU ready timeand all management workloads will slow down.

n Ensure that the allocated memory is supported by physical RAM inthe hardware. This rule requires that all allocated memory run onphysical RAM so that ballooning and swapping never occurs in theESXi Host.

n Ensure that management workloads run on a separate group ofdisks compared to any template or PlantCruise installation virtualmachine. This rule separates the critical management virtualmachines disk usage from any virtual machines used for stagingan operating system installation or PlantCruise installation. Thisseparation ensures that high disk requests and throughputdemanded by the installations do not affect the managementworkload performance. A simple approach to follow for this is tobuild a RAID group for management workload and a separateRAID group for staging virtual machine installations.

n Ensure that the disk storage capacity is adequate for the virtualmachines planned to run and also stored on the managementESXi host. Never plan to use more than 70% of the total datastorespace.

Supported management workloads

The supported management workloads are:

n Domain controller

n vCenter Server Appliance, which includes VMware UpdateManager

n Management client (optional)

n PlantCruise Backup and Restore (EBR)

n VMware Site Recovery Manager

Domain controller

Including the domain controller in the management host serves thepurpose of providing domain controller redundancy for the system(as other domain controllers are required) and reducing the cost ofproviding hardware for this required node. If domain infrastructurealready exists and the management network has the required access,the domain controller may not be required on the management host.

76


77

Ensure that the domain intended for vCenter Server is a member exitsbefore vCenter Server is prepared.

vCenter Server Appliance

The vCenter Server Appliance is a pre-configured Linux virtualmachine, which is optimized to run VMware vCenter Server and theassociated services. This appliance is central to the management ofthe virtual infrastructure and allows the creation, configuration,management and control of the virtual infrastructure.

Using vCenter Server, you can manage the ESXi hosts and their virtualmachines from a single user interface. vCenter Server consists of aserver component and an agent running on each ESXi host. Theserver component, vCenter Server, runs on a management node andcontains a database to store the configuration and performance data.The vCenter Server database must reside on the vCenter Server node.Each ESXi host includes a vCenter Agent that provides the requiredcommunication between each ESXi host and the vCenter Server.

Management Client

A management client virtual machine is in addition to the physicalmanagement client requirement. This virtual machine should run theWindows operating system and can be used for the creation of anESIS virtual hard disk and Utility virtual hard disks, which are used forthe installation of PlantCruise virtual machines.

A management client virtual machine is an optional requirement thatcan be used for the primary connected virtual machine when usingthe vSphere Web Client to manage and configure the virtualenvironment. It may be required when there is a desire to restrictinteractive access to the vCenter Server for security purposes.

When connecting to the vCenter Server, the vSphere Web Clientauthenticates access to the vCenter Server by way of Windowsauthentication. If you are using the management client to connect toan ESXi host directly, the vSphere Web Client authenticates access byway of a local ESXi host account.

TIP: The content and procedures in this guide are based on theusage of the Flash-based vSphere Web Client.


Configuring a web browser for use with the vSphereWeb Client

Honeywell recommends using the vSphere Web Client (Flash) with thevCenter Server Appliance (VCSA) version 6.5, and that you use thenative web client when connecting directly to ESXi servers.

Using the vSphere Web Client (Flash) involves launching yourbrowser, navigating to the ESXi host or vCenter's address, andchoosing to use the vSphere Web Client (Flash).

Minimum browser requirements

For information about the minimum requirements for browsers, see"vSphere Web Client Software Requirements" in the vSphere UpgradeGuide, Honeywell recommends using either Microsoft's InternetExplorer or Google's Chrome browsers.

After logging in for the first time, error dialogs about certificates mayappear. Follow the instructions in "Log in to vCenter Server by Usingthe vSphere Web Client" in the vCenter Server and Host ManagementGuide.

For more information about configuring web browsers, see"Configuring your browser" on page 140.

Experion Backup and Restore (EBR)

The EBR node is deployed into the management host and connects tothe management network to perform backup and recovery tasks. ANAS device is used as the de-duplication store for this appliance andthis NAS device is also connected to the management network. Formore information about the planning and configuration of EBR, seethe related topics.

For more information about the requirements for the virtualmachines that reside on the Management Host, see the HPSVirtualization Specification.

Using the management host to stage the installationof production workloads

The management host can be used to provide a part of the virtualinfrastructure where PlantCruise virtual machines and operatingsystem templates can be created and stored. This ESXi host is ideal as

78


79

an installation platform, as it segregates the installation process awayfrom potential disk I/O effects on the process workloads and stopspotential interference on the process networks. To accommodate forthis type of workload, consideration needs to be made where youselect the management host hardware. Define the installationscenarios and template creation scenarios that your system requires,and provide enough resources on the management host tosuccessfully accomplish this. As there is limited storage on amanagement host, you cannot stage or store the virtual machines ofan entire PlantCruise system. You should move staged PlantCruisevirtual machines to the required production ESXi host or a NASdevice.

Do not stage PlantCruise virtual machine installations on liveproduction ESXi hosts.

To successfully stage PlantCruise virtual machine installations on themanagement host:

n One or more virtual switches configured with the same name andsetting as the production ESXi hosts. These virtual switches areconfigured with no up-links.

n Separate disk groups for management workload and installationworkload to protect the management workloads from diskcontention. Use a separate build/installation volume. Do not usethe management volume as a storage repository.

n DVD media converted to ISO format for operating systems andPlantCruise installation. These ISO files are uploaded to thebuild/installation volume on the management host.

n Resource pools for the installation virtual machines configuredwith CPU limits to protect the management workloads.

Planning the management network in a DCSarchitectureThe management network spans the entire system in order to providemanagement access to all elements of the virtualizationinfrastructure.

In this context, virtual infrastructure refers to the basic elementsneeded to enable consolidation of workload (virtual machines) one ormore hardware platforms together with the means to manage thosevirtual machines and the hosting hardware. Specifically, the virtualinfrastructure includes:


n ESXi hosts

n Hypervisor installed on each ESXi host

n Virtual machines

n Virtual infrastructure management tools

n Management User Interfaces

n Backup Devices

At the core of the virtual infrastructure is the virtual infrastructuremanagement tools, which are used to:

n configure the virtualized system

n create virtual servers and desktops

n provision of virtual machines to hosts

n start virtual machines and monitoring their health

n monitor the health of hosts

n backup and restore virtual machines to/from backup devices

n facilitate the update of each element of the virtual infrastructure

The management network is used to perform these tasks. Adding themanagement network to a system to serve the needs of the FTEnetworks at the production level as well as those at the applicationlevel requires special considerations.

Planning the management network for usage withmultiple plant levels

A multi-plant level PlantCruise system is typically deployed with aLevel 2 subnet with a minimum of two FTE switches for each FTEcommunity, Level 3 subnet and Level 3 router, and a firewall andadditional routers that restricts connection to and from Level 3.5 andthe enterprise level.

The management network is inserted between the FTE productionnetwork (Level 2) and the application network (Level 3). Themanagement network is a dual one gigabyte network that isconnected to the production and application levels of the system witha redundant set of multi-layer switches with routing capabilities.

The ESXi hosts, management clients, and backup devices are connectdirectly to the management network. Each ESXi host, regardless ofthe plant level location is also directly connected to the management

80


81

network. This connection for ESXi hosts is in addition to the requiredproduction network connections (that is, FTE at Level 2).

Physical nodes that are used for production at Level 2, Level 3 orLevel 3.5 have no connection to the management network, and aretherefore, are not managed by, nor visible to the virtual infrastructuremanagement tools.

Dual management network

Dual management networks are supported for DCS architecturenetworks.

This requires the following:

n a separate pair of network interface ports on each ESXi host

n a separate pair of multi-layer switches with routing capabilitieswith crossover cable

n cabling

In a multiple plant level system, the dual management network can beimplemented between the supervisory level (Level 2) and theapplication level (Level 3). This deployment is referred to as Level 2.5.

Planning the management network in a SCADAarchitectureA single management network is supported for SCADA architecturenetworks.

This requires a separate pair of network interface ports on each ESXihost and a separate management network switch and cabling. Boththe production and management networks are routed to a gatewayrouter that enables secured access to business network (WAN).

Figure 6-3: Example SCADA architecture topology with a singlemanagement network


82


83

Planning how to organize the vCenter ServerHosts and their virtual machines should be organized within vCenterServer based on logical groupings.

These logical grouping will be different for each site depending on thescale of the virtualization environment and topology.

VMware inventory objectsVMware inventory objects include datacenters, folders, hosts, andvirtual machines. You can organize these inventory objects into ahierarchy to help monitor and manage the VMware virtualizationenvironment.

Object Description

datacenter A datacenter is the primary container of inventory objects, such as hostsand virtual machines. A vCenter Server can contain multiple datacenters.

For large virtualization implementations, datacenters can be used torepresent organizational units within the enterprise. In addition toproviding a container, the datacenter also serves as a boundary whenusing advanced features including vMotion.

For Experion virtualizations, a datacenter is used to contain all of theExperion virtual machines.

folder A folder is a container to further refine the grouping of inventoryobjects, for example, to group objects based on a physical location.Folders can also be used to assign security permissions. Inventoryobjects placed within a folder have the same permissions as the folder.

For large virtualization implementations, folders can be used to groupdatacenters, and then within datacenters, folders can be used to grouprelated ESXi hosts for an Experion cluster or system. You can usefolders to group hosts into any logical grouping that suits yourorganization.

ESXi host A host is a computer that is running ESXi virtualization software to runvirtual machines.

Hosts provide the CPU and memory resources that the virtual machinesuse and give virtual machines access to storage and network resources.Multiple virtual machines can run a host at the same time.


There are other inventory objects, such as clusters and resourcepools. For more information about these inventory objects, see the“vSphere Managed Inventory Objects” section of vSphere Server andHost Management and the Virtualization with Premium Platformguide..

Datacenter organization guidelinesThe recommended steps for creating an organization hierarchy withinvCenter Server (within the Home > Inventory > Hosts and Cluster view) is:

1. Create a datacenter.

2. Create and organize folders within the datacenter.

3. Add the ESXi hosts to the folders.After you add an ESXi host, you configure the ESXi host networksettings, and then you can create virtual machines on the ESXihost.

Organizing the datacenter

Folders should be used to group like components within the systems,for example, hosts, virtual machines, templates, and so on. Whennaming a folder, give it a name that clearly identifies this logicalgrouping. As your virtualization environment grows over time, specifichardware can be more easily found in its folder as opposed to havingto search through all generic names to find a given virtual machine orhost. You can create folders within other folders to further refinethese groupings.

ATTENTION: ESXi hosts should be allocated to either on-process usage or off-process usage.

84


85

Planning for time synchronizationTime synchronization requires an NTP server to distribute timethroughout the virtual infrastructure.

An NTP server could reside outside of the virtual infrastructure, or itcould be a virtual machine. All virtual machines synchronize timefrom this NTP server.

Time synchronization considerations

Identify the NTP server that will be the main timekeeper for the virtualenvironment. The NTP server could be a physical piece of hardware,such as a computer or switch, or it could be the primary PlantCruiseserver. You may already have an established NTP source for servingtime to the control hardware.

Ensure that the identified NTP server can be accessed by the virtualmachines.

Using the information below, determine the time synchronizationscenario based on whether you are using a Windows workgroup ordomain:

Windows domain

Set up time synchronization by following the instructions in the“Setting up time synchronization in a Windows domain checklist”topic in the “Setting up time synchronization” chapter of theSupplementary Installation Tasks Guide.

Consider if your Domain controllers are virtual machines or physicalmachines as this will change the requirement for an external timesource. When all domain controllers are virtual machines an externaltime source is required as the local clock on the virtual machine willdrift. The following VMware white paper gives more detail Virtualizinga Windows Active Directory Domain Infrastructure.

When you have a domain controller running as a physical machinethe local clock of this machine can provide a time reference that canbe used for the domain. This domain controller will need to be set asthe PDC emulator. In this scenario a Windows server uses the localclock as the time reference, the value of Root Dispersion for NTP replysent back to the ESXi Host will have a value too high for the ESXi hostto use. Changing the following Registry setting in the Windows serverfrom its default value of 10 to a new value of 0 will adjust the root


dispersion value to a value that will allow the ESXi hosts tosuccessfully synchronize time:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\"LocalClockDispersion"

ATTENTION: The ESXi hosts must synchronize with the domaincontroller or the external time source that the domaincontroller synchronizes with.

Workgroup without an external time source

Set up time synchronization by following the instructions in the“Setting up time synchronization in a workgroup without an externaltime source checklist” topic in the “Setting up time synchronization”chapter of the Supplementary Installation Tasks Guide.

When using virtual machines as the authoritative time source, timedrift can occur and this practice is not recommended.

ATTENTION: The ESXi hosts must synchronize with theauthoritative root server. In this scenario a Windows server usesthe local clock as the time reference, the value of RootDispersion for NTP reply sent back to the ESXi Host will have avalue too high for the ESXi host to use. Changing the followingRegistry setting in the Windows server from its default value of10 to a new value of 0 will adjust the root dispersion to a valuethat will allow the ESXi hosts to successfully synchronize time:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\"LocalClockDispersion"

Workgroup with an external time source

Set up time synchronization by following the instructions in the“Setting up time synchronization in a workgroup with an external timesource checklist” topic in the “Setting up time synchronization”chapter of the Supplementary Installation Tasks Guide.

ATTENTION: The ESXi hosts must synchronize with the externaltime source.

86


87

Planning to maintain the VMware environmentYou need to consider how to maintain and update the VMwarecomponents within a virtualization environment.

About vSphere Update ManagerWith vSphere Update Manager (VUM), you can automate patchmanagement and eliminate manual tracking and patching of ESXihosts.

Honeywell recommends that you use vSphere Update Manager toadminister and automate the application of patches and updates toESXi hosts.

You can use vSphere Update Manager to:

n Apply patches to ESXi hosts.

n Upgrade ESXi hosts to new releases.

n Upgrade VMware Tools within virtual machines.

n Upgrade virtual machine hardware versions.

n Upgrade virtual appliances.

For applying patches to Windows guest operating systems, Honeywellrecommends that you use the existing techniques for applyingWindows operating system updates to virtual machines. ThePlantCruise best practices for installing Honeywell-qualified Microsoftupdates are described in “Microsoft Security Updates and ServicesPacks” of the Network and Security Planning Guide.

VMware Tools and virtual machine hardware version upgradestypically require the virtual machine to be restarted. Therefore, youshould coordinate the remediation of these upgrades to occur withthe installation of Honeywell and qualified Microsoft updates as partof normal periodic maintenance

For more information about planning a Update Manager deployment,see Installing and Administering VMware vSphere Update Manager.

Considerations for VMware maintenance when usinglocal storageTo remediate or apply a patch or upgrade requires the ESXi host to beplaced into maintenance mode.


Maintenance mode requires all virtual machines on the ESXi host tobe moved to another ESXi host using vMotion, or for the virtualmachines on the ESXi host to be shutdown. vMotion is not supportedwhen the virtual machines are stored on the local storage of an ESXihost. Therefore, to remediate or apply a patch or upgrade to an ESXihost with local storage requires all virtual machines on the ESXi hostto be shutdown.

If on-process patching or upgrading of an ESXi host is required, theissue of all virtual machines on the ESXi host to be shutdown at thesame time must be considered when assigning PlantCruise virtualmachines to an ESXi host. To perform an on-process patch orupgrade of an ESXi host, virtual machines, or virtual appliances (VAs)requires a predefined process or plan. The process or plan mustclearly specify the shutdown and restart order of virtual machines,VAs, and ESXi hosts that support the running of the process.

Honeywell recommends that ESXi host patches and upgrades bedone during planned outages if virtual machines are stored on thelocal storage of an ESXi host.

Update Manager cannot be used to patch or upgrade an ESXi host ifit hosts a local storage vCenter Server. All virtual machines on theESXi host must be shutdown before it can be patched or upgraded.Shutting down vCenter Server will shutdown the Update Manager.

The ESXi host of a local storage virtual vCenter Server must bepatched or upgraded manually.

Considerations about upgrades, patches, and updatesUnderstand the differences between upgrades, patches, and updates.

What are upgrades?

An upgrade is a VMware version or point release change. For example,an upgrade is required to go from version 6.0 to 6.5.

An upgrade affects multiple components of the VMwareinfrastructure, including:

n vCenter Server Appliance with integrated Update Manager

n ESXi hosts

n VMWare Tools

n Virtual machine hardware versions

88


89

To maintain communication between these components during theupgrade process, the upgrade must be completed in a specific order.It is important to use VMware upgrade documentation to perform theupgrade. In addition, it is also important to keep in mind the localstorage special considerations during the upgrade process.

What are patches?

A patch contains one or more cumulative bulletins and patches aspecific VMware version. For example, a patch name is ESXi600-201711101.

The patch details include information on the action required on theESXi host and hosted virtual machines. For example, if the ESXi hostneeds to be restarted, if the hosted virtual machines need to beshutdown.

Patches typically affect a single component so they can be appliedwithout the concern of losing communications with othercomponents.

What are updates?

An update is a bulletin within a patch that contains a roll-up ofpatches for a release. For example, 6.5 Update 2 includes all of theversion 6.5 patches up to the date when 6.5 Update 2 was released.


Planning security for the VMware environmentYou need to consider how to implement security within the VMwarevirtualization environment.

User accounts, roles, and permissionsVMware user accounts should not be created within vCenter, insteadthe accounts and groups should be configured on the Windows ActiveDirectory domain controller. Within vCenter you can then define rolesand configure appropriate permissions for these roles and assignWindows users and/or groups to these roles.

vCenter Server users

Honeywell recommends that user accounts be maintained withinActive Directory on a Windows domain. It is also recommended thatyou use the Honeywell High Security Policy, and assign users to theHigh Security Policy global groups. You can then assign globalgroups to vCenter roles. For more information about the HoneywellHigh Security Policy, see the “Honeywell High Security Policy” topic inthe “Securing access to the Windows operating system” chapter of theNetwork and Security Planning Guide.

When planning for virtualization, you need to identify the rolesrequired to meet your security needs, and the permissions requiredfor each role.

Honeywell recommends that you create the following roles in vCenter:

Role Responsibilities

VirtualizationAdministrator

l Manages the Datacenter and the provision ofresources.

l Creates and configures the virtual machines.

l Creates and maintains the network configuration.

l Manages updates to the ESXi hosts and virtualmachines.

l Defines security for the ESXi host.

l Manages other virtual appliances.

PlantCruise Virtualizationl Manages PlantCruise virtual machines.

90


91

Role Responsibilities

Administratorl Manages patches to virtual machine guest operating

system and PlantCruise software.

l Creates and configures PlantCruise virtual machines.

l Starts up and shuts down PlantCruise virtualmachines.

PlantCruise VirtualizationUser

Starts up and shuts down PlantCruise virtual machines.

For more information about roles and permissions, including thehierarchical inheritance of permissions and details about eachpermission, see the following VMware references:

n "vSphere Permissions and User Management Tasks" in vSphereSecurity

n "Defined Privileges" in vSphere Security

Roles and PlantCruise security groups

Honeywell recommends that the following users or global groups beassigned the following roles:

Assign the following vCenterrole

To the following user or global group

Virtualization Administrator System administrator users.

PlantCruise VirtualizationAdministrator

Honeywell Administrators or Product Administratorsglobal group.

PlantCruise Virtualization User Local Engineers global group.

ESXi host users

Each ESXi host has two default users:

n The root user has full administrator privileges.

n The vpxuser is a user created only when the ESXi host is being


managed by a vCenter Server, and is the account used by vCenterServer to manage activities on that ESXi host.

About host lockdown modeHost lockdown mode prevents remote users from logging into anESXi host using the root user and password.

Host lockdown mode must be enabled to restrict direct access to theESXi host. That is, remote access to the ESXi host through the ESXiweb interface, the remote command-line interface (CLI), and accessthrough the Virtual Infrastructure (VI) API are not allowed for the rootuser.

ATTENTION: The root user is still authorized to log in to thedirect console user interface when host lockdown mode isenabled in Normalmode.

Host lockdown mode does not affect how other users access virtualmachines through the vSphere Web Client or Remote DesktopConnection (RDC).

Security hardeningIn addition to guidelines in the Virtualization with Premium Platformguide, Honeywell recommends that you follow these guidelines fromVMware when implementing your virtual infrastructure:

In a continuation of the review VMware performed of their hardeningguidance in vSphere 6.0, VMware have further reviewed all the itemsin vSphere 6.5 and removed many items as they are now the defaultand/or are no longer changeable or relevant. They've also markedeach item in the guide as to whether it is a "hardening" item or not,and as such there are now far fewer "Hardening" settings to consider,although Honeywell recommends reviewing all the settings.

For more information, seehttps://blogs.vmware.com/vsphere/2017/04/vsphere-6-5-security-configuration-guide-now-available.html.

The security hardening guidelines for vSphere 6.5 are detailed in thevSphere 6.5 Security Configuration Guide, and cover three generaloperating environments:

n All Environments - Profile 3

n Sensitive Environments - Profile 2

n Highest Security Environments - Profile 1

92


https://blogs.vmware.com/vsphere/2017/04/vsphere-6-5-security-configuration-guide-now-available.html

https://blogs.vmware.com/vsphere/2017/04/vsphere-6-5-security-configuration-guide-now-available.html

93

For Experion virtualized deployments, you should consider off-process systems as being in the Profile 3 operational environment andon-process systems should be considered as being in the Profile 2 orProfile 1 operational environments. Due to other recommendationsregarding the structure of your virtual infrastructure, some of theguidelines in VMware's vSphere 6.5 Security Configuration Guidemaynot be applicable or suitable for Experion systems. Details for theseare specified below. The guide only provides guidance for “ESXi”,"VM", and “vNetwork”.

The vSphere 6.5 Security Configuration Guide provides details on howto apply and how to audit for the item. In addition, VMware's vRealizeOperations (vROps) can be used to check the status of your systemwith the programmatic guidance items.

ESXi

For ESXi, Honeywell recommends following all guidelines (includingfor Profile 1) for the profile you choose to run on your system.

NOTE: Ensure SNMP is disabled unless you have an appropriateSNMP infrastructure.

VM

For VMs, Honeywell recommends following all guidelines (includingfor Profile 1) for the profile you choose to run on your system.

vNetwork hardening

For ESXi, Honeywell recommends following all guidelines.

Planning to backup the virtual infrastructureand virtual machines

Plan for the use of Experion Backup and Restore (EBR) to performbackups of the virtual infrastructure and virtual machines.

Virtual machine and virtual infrastructure backup is achieved throughtwo different mechanisms:


n EBR: This is the primary backup tool that allows virtual machinebackups to be moved out of the virtual infrastructure and ontoseparate storage as a system image. This data can also be movedto different physical locations for Disaster recovery if required.This is the same mechanism used to backup physical machines.

n Virtual Machine Replication: uses VMware vSphere Replication toreplicate virtual machines to different ESXi hosts. This method isuseful for virtual machines running applications that areinherently non-redundant and require rapid recovery. This backupmethod can be used in addition to EBR as a backup method, but isnot a substitute.

The following table outlines scenarios where EBR, Virtual machinereplication, or no action should be used when virtual machines andESXi hosts are powered off.

Scenario EBR ReplicationNoBackupAction

Comment

Hardware failure,repaired within 1hour and all datalost

X Use EBR and restore directly torepaired Host

Hardware failure,repaired within 1hour and no datalost

X

Hardware failure,replace host within1 hour,

X Use EBR and restore directly tonew host.

Hardware failure,repair longer than 1hour and no datalost.

X

Hardware failure,repair longer than 1hour and all datalost.

X X Run replication VMs immediatelyand transition back to originalVMs after using EBR to restore tothe repaired ESXi host.

Hardware failure,replacement hosttakes longer than 1

X X Run replication VMs immediatelyand transition back to originalVMs after using EBR to restore

94


95

Scenario EBR ReplicationNoBackupAction

Comment

hour to new ESXi host.

Hypervisor Update X

Virtual Machinecomplete failure inGuest Operatingsystem.

X Use EBR to restore single virtualmachine directly to Host.

Virtual Machine lossof files.

X Use EBR to restore a file from abacked up virtual machine.

Planning for EBR

EBR should run on the management ESXi host, so as not to putadditional load on the production ESXi hosts.

ATTENTION: The backup storage device must be installed andoperational before installing EBR.

If you are using shared storage, ensure the destination location forbackups are on different disk arrays to the virtual machines beingbacked up. This will ensure that an array failure will not lose theoriginal virtual machines and the backups.

Backup storage device considerations

Honeywell recommends use of a file share on a Network AttachedStorage (NAS).

The following list summarizes the planning considerations for eachbackup storage type. Refer to the EBR documentation for furtherplanning considerations.:

n Network file share on a NASSmaller backup location sizes are allowed. The NAS must beaccessible from the management network.


Planning for disaster recoveryDisasters do happen, so it is important that critical managementworkloads containing your virtualization infrastructure, along withyour production workloads, are backed up.

Backing up your virtualization infrastructure

The following nodes need to be considered in any backup plan:

n vCenter Server Appliance (VCSA)This virtual machine will be vital in restoring the rest of your virtualinfrastructure in the event that it is lost.

n Domain controllerAt least one domain controller should be backed up. In the eventthat all domain controllers are lost this one can be restored andother domain controllers can be built from scratch andsynchronized with this one.If at least one domain controller survives the disaster, domaincontrollers should not be restored from backup. Instead, a newdomain controller should be created.

n Anti-virus software, WSUS, and so onAny other management workload that contains important stateinformation.

For any of these machines that use Microsoft SQL Server, ensure thatthe Microsoft SQL VSS Writer is installed.

Ensure that VMware Tools is installed on all virtual machines foroptimal backup performance.

Off-site backup

The frequency that the backups are taken off-site is determined by theimportance of the data and systems. Any mission-critical systems ordata (those that must be recovered within minutes or hours of asystem failure) must be backed up and taken off-site daily, as aminimum. The location you choose is important. The backups mustnot only be safe and secure, but quickly and easily accessible in theevent of a disaster.

Honeywell recommends that you transfer your backup data onto aremovable media on a regularly-scheduled basis. You can store themedia off-site in a secure temperature-controlled environment for

96


97

long-term storage. In addition, you can define the backup retentionperiod for the media.

For details of how to create and restore backups on the NAS, see therelated topics.

Planning to replicate the virtual machinesVirtual machine replication is the process where a workload that isnot inherently redundant is automatically replicated onto a peer ESXihost. VMware vSphere Replication is used to replicate selected virtualmachines at a configured rate. In the event of a failure of an ESXihost, any replicated workload can be powered on and used until thefailure is rectified.

Virtual machine replication is primarily used for ACE virtual machinesin a PlantCruise system. However, methodologies could be applied toother non-redundant virtual machines if required. The ACE virtualmachines can be made available very quickly in the event of ahardware failure when an ESXi host is unavailable for an extendedperiod of time.

Note that on the Premium Platform, other technologies, such as FT,can be used to protect inherently non-redundant nodes such as theACE.


Figure 6-4: Example ACE workload replicated onto a differentproduction ESXi host

The virtual system illustrated above is an example of a system whereACE workload is replicated onto a different production ESXi Host andmaintained in a powered off state. The virtual machines with the suffix"- replica" in the name are the cloned virtual machines from the otherESXi Host. In this example "ACE 1" is running on ESXi Host 2 and thereplica of ACE 1 called "ACE 1 - replica" is not running but exists inthe inventory on ESXi Host 3. In the event of a hardware failure onESXi Host 2 the replica virtual machine "ACE 1 - replica" can bequickly powered on and this will allow the control strategy run by"ACE 1" to now be run by "ACE 1 - replica". When the hardware failureis rectified and the original virtual machine "ACE 1" is again available,the replica virtual machine "ACE 1 - replica" can be powered off then"ACE 1" can be powered on and the control strategy restarted on "ACE1".

ESXi Host resource requirements

Successful implementation of replication requires all ESXi Hosts thatare intended to run any replicated workload to have enough hardwareresources available for the anticipated failure scenarios. This isimportant so that resource contention does not occur. Eachreplicated virtual machine needs to be considered as a normalworkload on the ESXi Host when determining the workloaddeployment.

Refer to “Determining the number of ESXi hosts and hardwarerequirements” to ensure that the hardware is provisioned correctly.

Refer to “Replicating a virtual machine” for implementation details.

98


CHAPTER

7 IMPLEMENTING NETWORKS FOR A DCSARCHITECTURE

In this section:

Implementing a non-redundant management network for a DCSarchitecture 100

Implementing a redundant management network for a DCSarchitecture 108

Implementing an optional network configuration whereadditional L2.5 management ports are required 116

Implementing a separate management network for a DCSarchitecture 119

99

Implementing a non-redundant managementnetwork for a DCS architecture

NOTE: : This section is only applicable to the Essentials Platform.

Prior to implementing the management hosts and virtualinfrastructure tools, the management network must be establishedand configured.

Shown below is the typical network topology for a multiple plant levelsystem with the management network at Level 2.5.

Figure 7-1: Example network topology with the non-redundantmanagement network at Level 2.5

Level 2 FTE SwitchesFTE Yellow Network (Level 2)

Management Network (Level 2.5)

Level 3 Router

Level 2.5 Router

DMZ Network (Level 3.5)Level 3.5 Router

Level 1 Control Firewall

C300 Controllers

Enterprise Network (Level 4)

FTE Green Network (Level 2)

Firewall

Level 4 Router

Management subnet routed to share Management Infrastructure at Level 2.5

Management subnet routed to share Management Infrastructure at Level 2.5

L3 Production Network

100

Chapter 7 - Implementing networks for a DCS architecture

101

Prerequisites

The procedures to establish the network levels other than Level 2.5are beyond the scope of this guide. The Experion Network BestPractices Guide provides recommendations and guidelines toconsider when establishing Level 1, Level 2, Level 3, Level 3.5 andLevel 4. Honeywell Network Services can be consulted for the properconfiguration of the router at Level 3.

The Fault Tolerant Ethernet Overview and Implementation Guide canalso be used to establish Level 1 and Level 2 networks.

The procedure to connect Level 2.5 to Level 3 requires theestablishment of Level 3 network including the configuration of theLevel 3 router(s).

The procedure to connect Level 2 to Level 2.5 requires theestablishment of Level 2 (FTE) including the configuration of FTEswitches for each FTE community.

The figure below illustrates the connection of Level 2.5 to Level 3 withthe use of a single Level 3 router connected to a single Level 2.5router. This option requires that the Level 2.5 router support routingredundancy. The configuration of the single L2.5 router isimplemented with dual redundancy. This approach enables usage ofthe Honeywell provided L2.5 configuration templates. No update isrequired if dual redundancy is physically implemented at some latertime.

From the figure below, note the following:

n Use of a single Level 2.5 router that supports routing redundancy.For example, Cisco Systems' Hot Standby Router Protocol (HSRP).

n Allocation of two Level 3 router ports in the same VLAN. This VLANis dedicated to the connection to the A and B Level 2.5 routers.One port will be unused at this time.

n Common definition of a VLAN in both Level 2.5 A and B routers forthe management network.


Figure 7-2: Connection of non-redundant Level 2.5 to Level 3 using asingle Level 3 router

FTE Level 2 Switch Yellow

Firewall

FTE Level 2 Switch Green

Level 2.5 Router A

L3 Router

NewVLAN

SharedAddress

indicates routed port

Management VLAN

SharedAddress

SharedAddress

Crossover defined in A Router with no physical

connection

An alternative approach is to use redundant Level 3 routers with aVLAN common between the two Level 3 routers. This would be similarto that which is illustrated above for the Level 2.5 Routers or the FTELevel 2 switches and would therefore include the use of a crossoverbetween the primary and secondary Level 3 routers.

The table below can be used as a guide to performing the Level 3 andLevel 2.5 router configuration tasks to establish the managementnetwork as shown in the figure above. This procedure assumes usageof the recommended Cisco switch-routers that support HSRP.

To implement the management network for a DCS architecture(configure the Level 2.5 router and connect it to the Level 3 router):

Stage Task Description

Definesubnetsrequiredby addingmanagementnetworkand Level2.5 HSRPsubnet.

1. Allocate four IP addressesin an unused subnet for useredundant IP routingbetween Level 2.5 and Level3. This assumes a singleLevel 3 router where Level2.5 connections aregrouped together on Level3 router with a VLAN.

IP addresses required to run with theHSRP in Level 2.5

1. Level 3 VLAN IP address

2. Unique IP address for Level 2.5 Arouter connection in Level 3 VLAN

3. Unique IP address for Level 2.5 Brouter connection in Level 3 VLAN

4. Subnet mask for Level 3 VLAN

5. Shared address for items 2 and 3

102


103


above.

2. Allocate unused subnetwith enough IP address toaccommodate allmanagement networkconnection requirements.

l IP addresses for the managementVLAN

a. Unique IP address for Level 2.5 Arouter connection.

b. Unique IP address for Level 2.5 Brouter connection (for future use).

c. Subnet mask for items 1 and 2above.

d. Shared address for items 1 and 2above.

l IP addresses for all othermanagement connected devices (ESXihosts, management client, NAS, andvDR).

Prepare toupdatethe Level3 routerconfiguration.

3. Allocate 1 physicalinterface (port) on therouter. Note that you maybe re-allocating an existinginterface that is currentlyused for Level 2 connectionto Level 3. Reserve 1physical interface (port) onthe router for future use.

Port Usage # GB Port Type

R S

Level 2.5A 1 0 Y

Level 2.5B 1 0 Y

O = Optional

Y = Yes

4. Define the ID of the newVLAN on the Level 3 router.

New VLAN ID

5. Be familiar with how toupdate the currentconfiguration of the Level 3router

Preparetoconfigurethe Level2.5 router.

6. Download the Level 2.5router A template and theLevel 2.5 router B templatefrom the HoneywellProcess Solutions web site

The templates define a configuration fora 24 port switch router from the Ciscofamily of switch routers.

Honeywell recommends usage of theport allocation scheme as defined in the



(http://www.honeywellprocess.com).

templates.

The default definition in theconfiguration files is for usage with non-stackable switch-routers. Modificationsare required for usage with stackableswitch-routers.

7. Validate that the IOS inthe Level 2.5 router meetsthe site requirements forminimum IOS version.

Consult with site policies regarding theminimum IOS version that should beinstalled in the Level 2.5 switch-router.

8. Define host name foreach router.

9. Allocate physicalinterfaces (ports) for eachLevel 2.5 router.

Note that FTE communityconnections are defined asthe top level of yellow sidefor the Level 2.5. (The topside of green side would beallocated on the 2nd routerif implemented in the future

Interface(Port)Usage

Quantity GB

RoutedPort

SwitchPort

1 Y Y

FTECommunities

1 percommunity

O Y

VirtualInfrastructureMgmtConnections

1 perconnection

Y Y

CrossoverCable

1 Y Y

O = Optional

Y = Yes

10. For each FTEcommunity, know the FTEIP addresses that are usedfor HSRP between Level 2.5and Level 2.

IP addresses for each FTE community:

1. Unique IP address for yellowconnection in FTE community subnet.

2. Unique IP address for green

104


105


connection in FTE community subnet.

3. Subnet mask for FTE Community.

4. Unique shared virtual address (defaultgateway) for FTE Community.

11. Define managementnetwork VLAN.

Define VLAN ID (this is pre-defined inLevel 2.5 template files as 401).

12. Review ACLs. Update to be site compliant.

13. Be familiar with how toset up and access the Level2.5 routers forconfiguration.

TIP: Loading the Level 2.5configuration files is similar to thesteps documented in section 9.7,“Configuring Cisco Switches” ofthe Fault Tolerant EthernetOverview and ImplementationGuide.

Completephysicalconnections.

14. Complete the physicalconnections according tothe interface (port)allocations established inthe router preparation tasksfor both Level 3 and Level2.5.

Use the information defined in tasks 3and 9.

Updatethe Level3 routerconfiguration.

15. Add VLAN for Level 2.5connections.

Use the information defined in tasks 1.1and 4.

16. Update the IP routesettings.

Add the IP address of the VLAN definedin task 1.5.

17. Add interface for uplinkfrom Level 2.5 A router.

Use the information defined in tasks 3and 4:

l Assign the interface to the VLANdefined in task 15.

l Do not assign an IP address.



l Disable any discover protocol.

18. Add interface for uplinkfrom Level 2.5 B router (forfuture use).

Use the information defined in tasks 3and 4:

l Assign the interface to the VLANdefined in task 15.



ConfigureLevel 2.5router.

19. Create configuration file Preferred method is to use the Honeywellprovided Level 2.5 templates (see task 6).These instructions assume usage of thepreferred method to create Level 2.5configuration file.

20. Add VLAN forManagement Network

Pre-defined in the A template (that is,VLAN 401) as its own spanning treeinstance.

No additional updates if the pre-definition is suitable for usage.

21. Add the interface for theLevel 3 router connection.

Use the information defined in tasks 1.2,1.4. 1.5 and 9.

Pre-defined in the template with HSRPactivated.

22. Add a interface for eachYellow FTE communityconnection.

Use the information defined in tasks 9and 10.1, 10.3, and 10.4.

23. Add the interface rangefor the Virtual Infrastructureconnections.


24. Add the interface for thecrossover cable.


25. Add the VLAN interfacefor the managementnetwork.

Use the information defined in tasks 2.1,2.3, 2.4 and 11 (for A router).

This is pre-defined in the template toactivate HSRP on the default gateway IPaddress of the management network.

106


107


26. Update the IP routesettings.

Add default route for the IP address ofthe Level 3 VLAN.

Use the information from task 1.1.

27. Copy the updatedconfiguration file to theLevel 2.5 router.

TIP: Loading the Level 2.5configuration files is similar to thesteps documented in section 9.7,“Configuring Cisco Switches” ofthe Fault Tolerant EthernetOverview and ImplementationGuide.

Confirmthat theLevel 2.5router isoperational

28. Validate the following:

1. No blocked ports.

2. Correct states forPrimary.

3. Protocol and interfaceare “active”

Use:

1. Show span.

2. Show standby (Primary is in activestate).

3. Show interface (Interface is Up,Protocol is Up).


Implementing a redundant managementnetwork for a DCS architecture


Shown below is the typical network topology for a multiple plant levelsystem with the management network at Level 2.5.

Figure 7-3: Example network topology with the redundantmanagement network at Level 2.5


C1 FTE Yellow Network (Level 2)


Level 3 Routers

Level 2.5 Routers



C300 Controllers


C1 FTE Green Network (Level 2)

Firewall

Level 4 Router

Cn FTE Green Network (Level 2)

Cn FTE Yellow Network (Level 2)


C300 Controllers

Management Subnet routed to share Management Infrastructure at Level 2.5

Management Subnet routed to share Management Infrastructure at Level 2.5


Application Network (Level 3)


108


109

Prerequisites

The procedures to establish the network levels other than Level 2.5are beyond the scope of this guide. The Experion Network BestPractices Guide provides recommendations and guidelines toconsider when establishing Level 1, Level 2, Level 3, Level 3.5 andLevel 4. Honeywell Network Services can be consulted for the properconfiguration of the router at Level 3.



The procedure to connect Level 2 to Level 2.5 requires theestablishment of Level 2 (FTE) including the configuration of FTEswitches for each FTE community.

The figure below illustrates the connection of Level 2.5 to Level 3 withthe use of a single Level 3 router connected to a pair of Level 2.5routers. This option requires that the Level 2.5 routers support routingredundancy.


n Use of redundant Level 2.5 routers that support routingredundancy. For example, Cisco Systems’ Hot Standby RouterProtocol (HSRP).

n Allocation of two Level 3 router ports in the same VLAN. This VLANis dedicated to the connection to the A and B Level 2.5 routers.

n Common definition of a VLAN in both Level 2.5 A and B routers forthe management network.


Figure 7-4: Connection of redundant Level 2.5 to Level 3 using a singleLevel 3 router


The table below can be used as a guide to performing the Level 3 andLevel 2.5 router configuration tasks to establish the managementnetwork as shown in the figure above. This procedure assumes usageof the recommended Cisco switch-routers that support HSRP.

To implement the management network for a DCS architecture(configure the Level 2.5 router and connect it to the Level 3 router):


Definesubnetsrequired byaddingmanagement networkand Level2.5 HSRPsubnet.

1. Allocate four IPaddresses in anunused subnet foruse redundant IProuting betweenLevel 2.5 and Level3. This assumes asingle Level 3 routerwhere Level 2.5connections are

IP addresses required to run with the HSRP inLevel 2.5


2. Unique IP address for Level 2.5 A routerconnection in Level 3 VLAN

3. Unique IP address for Level 2.5 B routerconnection in Level 3 VLAN


110


111


grouped togetheron Level 3 routerwith a VLAN.

5. Shared address for items 2 and 3 above.

2. Allocate unusedsubnet with enoughIP address toaccommodate allmanagementnetwork connectionrequirements.

l IP addresses for the management VLAN

a. Unique IP address for Level 2.5 A routerconnection.

b. Unique IP address for Level 2.5 B routerconnection.

c. Subnet mask for items 1 and 2 above.

d. Shared address for items 1 and 2 above.

l IP addresses for all other managementconnected devices (ESXi hosts, managementclient, NAS, and vDR).

Prepare toupdate theLevel 3routerconfiguration.

3. Allocate 2physical interfaces(ports) on the routerNote that you maybe re-allocating anexisting interfacethat is currentlyused for Level 2connection to Level3.


R S

Level 2.5A 1 0 Y

Level 2.5B 1 0 Y

O = Optional

Y =-Yes

4. Define the ID ofthe new VLAN onthe Level 3 router.

New VLAN ID

5. Be familiar withhow to update thecurrentconfiguration of theLevel 3 router

Prepare toconfigurethe Level2.5 A and Brouters.

6. Download theLevel 2.5 router Atemplate and theLevel 2.5 router Btemplate from the

The templates define a configuration for a 24port switch router from the Cisco family ofswitch routers.

Honeywell recommends usage of the portallocation scheme as defined in the templates.



Honeywell ProcessSolutions web site.

The default definition in the configuration filesis for usage with non-stackable switch-routers.Modifications are required for usage withstackable switch-routers.

7. Validate that theIOS in the Level 2.5A router is the sameversion as the IOS inthe Level 2.5 Brouter.

Ensure that each switch router is installed withthe same IOS version to avoid communicationproblems.

Consult with site policies regarding theminimum IOS version that should be installedin each Level 2.5 switch-router.

8. Define host namefor each router.

9. Allocate physicalinterfaces (ports)for each Level 2.5router.

Note that FTEcommunityconnections aredefined as the toplevel of yellow sidefor the Level 2.5 Aand top side ofgreen side for Level2.5 B.


Quantity GB

Routed Port

SwitchPort

Level 3 1 O Y

FTECommunities

1 percommunity

O Y

VirtualInfrastructure MgmtConnections

1 perconnection

Y Y

CrossoverCable

1 Y Y

O = Optional

Y =-Yes

10. For each FTEcommunity, knowthe FTE IPaddresses that areused for HSRPbetween Level 2.5and Level 2.


1. Unique IP address for yellow connection inFTE community subnet.

2. Unique IP address for green connection inFTE community subnet.

112


113




11. Definemanagementnetwork VLAN.

Define VLAN ID (this is pre-defined in Level 2.5template files as 401).


13. Be familiar withhow to set up andaccess the Level 2.5routers forconfiguration.

TIP: Loading the Level 2.5 configurationfiles is similar to the steps documented insection 9.7, “Configuring Cisco Switches”of the Fault Tolerant Ethernet Overviewand Implementation Guide.


14. Complete thephysicalconnectionsaccording to theinterface (port)allocationsestablished in therouter preparationtasks for both Level3 and Level 2.5.

Use the information defined in tasks 3 and 9.

Update theLevel 3routerconfiguration.

15. Add VLAN forLevel 2.5connections.

Use the information defined in tasks 1.1 and 4.

16. Update the IProute settings.

Add the IP address of the VLAN defined in task1.5.

17. Add interface foruplink from Level2.5 A router.

Use the information defined in tasks 3 and 4:

l Assign the interface to the VLAN defined intask 15.





18. Add interfacefor uplink fromLevel 2.5 B router.





ConfigureLevel 2.5 Arouter.

19. Createconfiguration file

Preferred method is to use the Honeywellprovided Level 2.5 templates (see task 6). Theseinstructions assume usage of the preferredmethod to create Level 2.5 configuration file.

20. Add VLAN forManagementNetwork

Pre-defined in the A template (that is, VLAN401) as its own spanning tree instance.

No additional updates if the pre-definition issuitable for usage.

21. Add theinterface for theLevel 3 routerconnection.

Use the information defined in tasks 1.2, 1.4. 1.5and 9.


22. Add a interfacefor each Yellow FTEcommunityconnection.

Use the information defined in tasks 9 and10.1, 10.3, and 10.4.

23. Add theinterface range forthe VirtualInfrastructureconnections.


24. Add theinterface for thecrossover cable.


25. Add the VLANinterface for themanagementnetwork.

Use the information defined in tasks 2.1, 2.3, 2.4and 11 (for A router).

This is pre-defined in the template to activateHSRP on the default gateway IP address of themanagement network.

114


115



Add default route for the IP address of theLevel 3 VLAN.


27. Copy theupdatedconfiguration file tothe Level 2.5 Arouter.

TIP: Loading the Level 2.5 configurationfiles is similar to the steps documented insection 9.7, “Configuring Cisco Switches”of the Fault Tolerant Ethernet Overviewand Implementation Guide.

ConfigureLevel 2.5 Brouter.

28. Repeat tasks 19through to 26 forthe Level 2.5 Brouter.

Predefined in the B template. Adjust the Level2.5 A tasks as follows:

l Task 21: Use the information defined in tasks1.3, 1.4, 1.5 and 9.

l Task 22: Use the information defined in tasks9 and 10.2, 10.3, and 10.4.

l Task 25: Use the information defined in tasks2.2, 2.3, 2.4, and 11 (for B router).

29. Copy theupdatedconfiguration file tothe B device

Confirmthat Level2.5 Routersareoperational

30. Validate thefollowing:

1. No blockedports.

2. Correct states forPrimary/Secondary.

3. Protocol andinterface are“active”

Use:

l Show span.

l Show standby (Primary is in active state,Secondary is in standby state).

l Show interface (Interface is Up, Protocol isUp).


Implementing an optional networkconfiguration where additional L2.5management ports are required

Depending on your system architecture you may need additionalManagement Network ports over those provided by your L2.5 router.In this instance you can connect additional 1GB switches tomanagement ports on the L2.5 Router A and B configured tocommunicate on the existing management network.

Shown below is the network topology recommended to beimplemented when additional ports are required in the managementnetwork over those available in the L2.5 routers.

116


117

Figure 7-5: Example DCS architecture network topology withredundant additional management network switches

Prerequisites

n A redundant management network already exists.

n You have the additional switch hardware available.


To implement an optional network configuration where additionalL2.5 management ports are required:


Configureexisting L2.5Managementrouters A & B

1. Set Spanningtree mst 0 and 4priorities in theL.2.5 routers

To make the blocking behavior of spanning treemore deterministic the priority of mst 0 and 4 areset to 4096 and 8192.

2. Router Apriority 4096

3. Router Bpriority 8192

4. Removespanning-treeportfast onuplink ports inthe L2.5 RouterA and B.

The Management network in the L2.5 routers isassigned to VLAN 401. Each additional switchrequires a uplink to both Router A and B. Thistasks modifies the selected ports to removespanning-tree portfast

Configuretheadditionalmanagementswitches A &B

5. Add VLAN401to the additionalManagementswitches

The existing L2.5 Routers Management network isdefined as VLAN401. The additional managementswitch must be configured to communicate onVLAN401 to match.

6. Assign all theswitch ports asuntaggedmembers ofVLAN401.

7. ConfigureVLAN 1 to beexcluded on allports.

8. Repeat steps 4– 7 for alladditionalmanagementswitches.

118


119

Implementing a separate management networkfor a DCS architecture


Shown below is the typical network topology for a multiple plant levelsystem with single management network at Level 2 that is connectedto Level 2.5 or Level 3. The primary difference between thisimplementation and that of the previous section is that themanagement network connection is limited to a single routed port onLevel 2.5 or Level 3 network layer. This implementation approach canbe used with a new or pre-existing Level 2.5 or Level 3 network layer.

Figure 7-6: Example network topology with management network atLevel 2 connected to Level 2.5



Management Network (Level 2)

Level 2.5 Routers

Level 3.5 Router


C300 Controllers



Level 4 Router




C300 Controllers

Management Switch A

Level 3 Routers





Management Switch B (optional )

Firewall

DMZ Network (Level 3.5)

Management Subnet routed to share management infrastructure at Level 3

120


121

Figure 7-7: Example network topology with management network atLevel 2 connected to Level 3



Level 3 Routers



C300 Controllers



Firewall

Level 4 Router




C300 Controllers



Management Switch A

Management Subnet routed to share management infrastructure at Level 3



Management Switch B (optional )

The procedures to establish the network levels other than Level 2.5are beyond the scope of this guide. The PlantCruise Network BestPractices Guide provides recommendations and guidelines toconsider when establishing Level 1, Level 2, Level 3, Level 3.5 andLevel 4. Honeywell Network Services can be consulted for the properconfiguration of the router at Level 3.




The procedure to connect Level 2 to Level 2.5 or Level 3 requires theestablishment of Level 2 (FTE) including the configuration of FTEswitches for each FTE community.

The figure below illustrates the connection of a single managementnetwork to redundant Level 2.5 routers. Also illustrated is theconnection from Level 2.5 to Level 3 with the use of a single Level 3router connected to redundant Level 2.5 routers. This option requiresthat the Level 2.5 router support routing redundancy. The Honeywellprovided L2.5 configuration templates can be used to configure L2.5routers. However, the templates assume implementation of themanagement network as a separate vLAN. See the table below todetermine the template items that can be ignored whenimplementing a separate management network.


n Use of redundant Level 2.5 or Level 3 routers that support routingredundancy. For example, Cisco Systems' Hot Standby RouterProtocol (HSRP).

n Allocation of two Level 3 router ports in the same VLAN. This VLANis dedicated to the connection to the A and B Level 2.5 routers (ifdeploying with L2.5 only).

n Allocation of a single router port in Level 2.5 Router A or Level 3Router A for the connection of the management switch.

n Option: Allocation of a single router port in Level 2.5 Router B orLevel 3 Router B for the connection of the management switch.

122


123

Figure 7-8: Connection of single management network to redundantLevel 2.5

Level 2 FTE Switch Yellow Level 2 FTE Switch Green

Level 2.5 Router A Level 2.5 Router B

Level 3 Router

NewVLAN

SharedAddress

SharedAddress

Firewall

Indicates routed port

Management Switch


The table below can be used as a guide to performing the Level 3 andLevel 2.5 router configuration tasks as well as the management switchconfiguration tasks to establish the management network as shown inthe figure above. This procedure assumes usage of the recommendedCisco switch-routers for use at Level 2.5 and the Level 2 managementswitch.

To implement a separate management network for a DCSarchitecture:


Definesubnetsrequired byaddingmanagemen

1. Allocate four IPaddresses in anunused subnet foruse redundant IProuting between

IP addresses required to run with the HSRP inLevel 2.5


2. Unique IP address for Level 2.5 A router



t networkand Level2.5 HSRPsubnet.

Level 2.5 and Level3. This assumes asingle Level 3router where Level2.5 connections aregrouped togetheron Level 3 routerwith a VLAN.

connection in Level 3 VLAN

3. Unique IP address for Level 2.5 B routerconnection in Level 3 VLAN


5. Shared address for items 2 and 3 above.

2. Allocate unusedsubnet with enoughIP address toaccommodate allmanagementnetwork connectionrequirements.

1. Subnet mask for Level 2 managementnetwork.

2. IP addresses for all other managementconnected devices (ESXi hosts,management client and NAS).

Prepare toupdate theLevel 3routerconfiguration.

3. Allocate 2physical interfaces(ports) on therouter Note that youmay be re-allocating anexisting interfacethat is currentlyused for Level 2connection to Level3.


R S

Level 2.5A 1 0 Y

Level 2.5B 1 0 Y

O = Optional

Y = Yes

4. Define the ID ofthe new VLAN onthe Level 3 router.

New VLAN ID

5. Be familiar withhow to update thecurrentconfiguration of theLevel 3 router

Prepare toconfigurethe Level2.5 A and Brouters.

6. Download theLevel 2.5 router Atemplate and theLevel 2.5 router Btemplate from theHoneywell Process

The templates define a configuration for a 24port switch router from the Cisco family ofswitch routers.

Honeywell recommends usage of the portallocation scheme as defined in the templates.

124


125


Solutions web site. The default definition in the configuration filesis for usage with non-stackable switch-routers.Modifications are required for usage withstackable switch-routers. The default definitionin the configuration files implements theredundant management network as a vLAN.Modifications are required to remove this vLANand are identified in the remaining tasks.

7. Validate that theIOS in the Level 2.5A router is the sameversion as the IOSin the Level 2.5 Brouter.

Ensure that each switch router is installed withthe same IOS version to avoid communicationproblems.

Consult with site policies regarding theminimum IOS version that should be installedin each Level 2.5 switch-router.

8. Define host namefor each router.

9. Allocate physicalinterfaces (ports)for each Level 2.5router.

Note that FTEcommunityconnections aredefined as the toplevel of yellow sidefor the Level 2.5 Aand top side ofgreen side for Level2.5 B. Allocate anadditional physicalinterface (port) inLevel 2.5 A for themanagement switchconnection.


Quantity GB

Routed Port

SwitchPort

1 O Y

FTECommunities

1 percommunity

O Y

Management Switch

1 Y Y

O = Optional

Y = Yes

10. For each FTEcommunity, knowthe FTE IPaddresses that areused for HSRP


1. Unique IP address for yellow connection inFTE community subnet.



between Level 2.5and Level 2.

2. Unique IP address for green connection inFTE community subnet.




12. Be familiar withhow to set up andaccess the Level 2.5routers forconfiguration.

TIP: Loading the Level 2.5 configurationfiles is similar to the steps documentedin section 9.7, “Configuring CiscoSwitches” of the Fault Tolerant EthernetOverview and Implementation Guide.


13. Complete thephysicalconnectionsaccording to theinterface (port)allocationsestablished in therouter preparationtasks for both Level3 and Level 2.5.


Update theLevel 3routerconfiguration.

14. Add VLAN forLevel 2.5connections.

Use the information defined in tasks 1.1 and 4.


Add the IP address of the VLAN defined in task1.5.

16. Add interfacefor uplink fromLevel 2.5 A router.





17. Add interface Use the information defined in tasks 3 and 4:

126


127


for uplink fromLevel 2.5 B router.




ConfigureLevel 2.5 Arouter.

18. Createconfiguration file

Preferred method is to use the Honeywellprovided Level 2.5 templates (see task 6). Theseinstructions assume usage of the preferredmethod to create Level 2.5 configuration file.

19. Remove VLANfor ManagementNetwork

Remove the following lines from theconfiguration file in order to eliminate the pre-defined vLAN for management in the Atemplate (that is, VLAN 401):vlan 401name VL-Management!! Configure L2.5 as spanning treeroot for management network !spanning-tree mstconfigurationrevision 1instance 4 vlan 401spanning-tree mst 4priority 4096

20. Add theinterface for theLevel 3 routerconnection.

Use the information defined in tasks 1.2, 1.4.1.5 and 9.


21. Add a interfacefor each Yellow FTEcommunityconnection.

Use the information defined in tasks 9 and10.1, 10.3, and 10.4.

22. Replace theinterface range forthe VirtualInfrastructure withthe interface for themanagement

Use the information defined in tasks 9. Removethe following lines from the configuration filein order to eliminate the Virtual Infrastructureinterfaces defined on vLAN:interface rangeGigabitEthernet 0/13 - 23



switch connection. description virtual managementnetworkswitchport access vlan 401switchport mode access spanning-treeportfast

Replace the removed lines with the following:interface GigabitEthernet 0/13descriptionManagement Network for VirtualInfrastructure at L2no switchportip address<ManagementNetwork Subnet Mask>no ip redirectsno ip unreachablesno ip proxy-arpno ip mroute-cache

Use the information in task 2.1 forManagement Network Subnet Mask

23. Remove theinterface for thecrossover cable.

Remove the following lines from theconfiguration file in order to eliminate the useof the crossover cable:interfaceGigabitEthernet0/24 descriptionvirtualmanagment networkcrossover cableswitchport access vlan 401switchport modeaccess

24. Add the VLANinterface for themanagementnetwork.

Remove the following lines from theconfiguration file in order to eliminate the pre-defined vLAN for management in the Atemplate (that is, VLAN 401):interface Vlan401description managment vlan ipaddressMgmtAddrA subnetmaskno ip redirectsno ip unreachablesno ip proxy-arp

128


129


no ip mroute-cacheip access-group 130 instandby 241 ip SharedVirtMgmtAddrstandby 241 timers 2 6standby 241 priority 105standby 241 preemptdelay minimum 90


Add default route for the IP address of theLevel 3 VLAN.


26. Copy theupdatedconfiguration file tothe Level 2.5 Arouter.

TIP: Loading the Level 2.5 configurationfiles is similar to the steps documentedin section 9.7, “Configuring CiscoSwitches” of the Fault Tolerant EthernetOverview and Implementation Guide.

ConfigureLevel 2.5 Brouter.

27. Repeat tasks 18through to 26 forthe Level 2.5 Brouter.

Predefined in the B template. Adjust the Level2.5 A tasks as follows:

l Task 20: Use the information defined in tasks1.3, 1.4, 1.5 and 9.

l Task 21: Use the information defined in tasks9 and 10.2, 10.3, and 10.4.

l Task 22: Perform only the part of task toremove lines. Since there is no managementswitch connection on Level 2.5 B router, skipthe part of task to add lines.

28. Copy theupdatedconfiguration file tothe B device

Confirm thatLevel 2.5Routers areoperational

29. Validate thefollowing:

l No blockedports.

l Correct states for

Use:

1. Show span.

2. Show standby (Primary is in active state,Secondary is in standby state).



Primary/Secondary.

l Protocol andinterface are“active”

3. Show interface (Interface is Up, Protocol isUp).

ConfigureManagement switch

30. Complete thephysicalconnections to theport allocationsestablished in task2.

31. Configure theswitch withrecommendedsettings.

1. Configure the global wide settings on theswitch, including:

l Enable rapid spanning tree (RSTP) OnCisco switches, this is MST or PVRSTP+.

l On Cisco switches, configure VLANtrunking protocol (VTP) modetransparent, which disables the autoVLAN configuration. Other switchmanufacturers may have a similarconfiguration setting.

l No ip gratuitous-arps.

l No ip domain-lookup.

2. Best practice recommendation is that youenable "spanning-tree portfast" on all non-uplink ports (ports that aren't connected toother switches) and enable "switchportmode access" on all ports.

3. The interfaces on the management networkthat connect to the ESXi host need to matchthe speed and duplex settings of the ESXihost physical network interface cards (NICs).

4. Best practice recommendation is thatunused ports are shutdown and placed in anunused VLAN.

130


CHAPTER

8 IMPLEMENTING NETWORKS FOR A SCADAARCHITECTURE

In this section:

Preparing a gateway router for a SCADA architecture 132

Preparing the production network for a SCADA architecture 133

Configuring the virtual production network for a SCADAarchitecture 134

Implementing the management network for a SCADAarchitecture 135

131

Preparing a gateway router for a SCADAarchitecture

Note: This section is only applicable to the Essentials Platform.

Prepare a gateway router to enable communication between theproduction and management networks, and to control the flow ofnetwork traffic between the production network and the businessnetwork (WAN).

The purpose of the gateway router is to:

n Enable communication between the production and managementnetworks. This communication should be well documented andlimited so that it protects the integrity and security of the overallnetwork.

n For virtualization environments that are part of a backup controlcenter (BCC) solution, the gateway router enables communicationbetween the storage and management networks for data thatmust cross through a wide area network (WAN).

You can use a multi-layer switch with routing capabilities as thegateway router.

Prerequisites

n Best practice is to place a firewall between the WAN and thegateway router.

n Network access control lists (ACLs) are configured on the gatewayrouter. For more information about configuring network accesscontrol lists (ACLs), see the Experion Virtualization: Network RouterConfiguration Best Practices whitepaper.

n Experion Virtualization: Network Router Configuration BestPractices whitepaper, which is available from the HoneywellProcess Solutions web site (http://www.honeywellprocess.com).

To prepare a gateway router

1. Define and configure the subnets and VLANs.

2. Connect the gateway router to the WAN through a firewall. Thisconnection is recommended to be a single point of connectivity.

3. Disable the proxy address resolution protocol (ARP) on the

132

Chapter 8 - Implementing networks for a SCADA architecture


133

gateway router.

4. The gateway router must be connected to the production,management, and storage switch interfaces that are configured asuplink ports.

5. Enable filtering as described in the Experion Virtualization: NetworkRouter Configuration Best Practices whitepaper.

Preparing the production network for a SCADAarchitecture

Prepare the production network, including installing the productionnetwork switch and network cabling, and configuring the productionnetwork switch (if not already installed and configured).

Prerequisites

n You have followed the manufacturer's instructions and setup theproduction network switch on the required subnet.

To prepare the production network

1. Configure global wide settings on the switch, including:

a. Enable rapid spanning tree (RSTP). On Cisco switches, this isMST or PVRSTP+.

b. On Cisco switches, enable VLAN trunking protocol (VTP) modetransparent, which disables the auto VLAN configuration.Other switch manufacturers may have a similar configurationsetting.

c. No ip gratuitous-arps.

d. No ip domain-lookup.

2. Configure the switch to meet any additional requirements of theselected SCADA devices.

3. Honeywell best network practices recommends that you enable“spanning-tree portfast” on all non-uplink ports (ports that aren’tconnected to other switches) and enable “switchport modeaccess” on all ports.

4. The interfaces on the production network that connect to the ESXi


host need to match the speed and duplex settings of the ESXi hostphysical network interface cards (NICs).

Configuring the virtual production network fora SCADA architecture

Configure the virtual production network on the ESXi host andconfigure the virtual switch to communicate with the physicalproduction NICs.

To configure the virtual production network

1. On the vSphere Web Client (Flash) (within the Home > Inventories >Hosts and Clusters view), locate the ESXi host. Click the Configuretab and then under the Networking group, click the Virtual Switchesitem.

2. Above the list of switches, click the Add Host Networking icon..The Networking wizard appears.

3. At step 1, Select connection type, select Virtual Machine Port Group fora Standard Switch and then click Next.The Select target device page appears.

4. Select New Standard Switch and then click Next.The Create a Standard Switch page appears.

a. In the Add Physical Adapters to the Switch dialog, select theNetwork Adapter to add, then click OK.

b. Click Next.If prompted with the warning All active physical networkadapters assigned to the switch are disconnected, click OK.

The Connection Settings page of the wizard appears.

5. Provide a Network Label, for example, Production Network.6. Leave the vLAN ID set to the default value of 0.7. Click Next.8. Set the following connection settings:9. Click Next.

10. Verify the details, and click Finish.

134


135

Implementing the management network for aSCADA architecture

Prior to implementing the management ESXi hosts and virtualinfrastructure tools, the management network must be establishedand configured. For SCADA architectures, consider the followingsettings when implementing the management network.

To implement the switch for the management networkin a SCADA architecture

1. Configure the global wide settings on the switch, including:

a. Enable rapid spanning tree (RSTP). On Cisco switches, this isMST or PVRSTP+.

b. On Cisco switches, configure VLAN trunking protocol (VTP)mode transparent, which disables the auto VLAN configuration.Other switch manufacturers may have a similar configurationsetting.

c. No ip gratuitous-arps.

d. No ip domain-lookup.

2. Honeywell recommends that you enable “spanning-tree portfast”on all non-uplink ports (ports that aren’t connected to otherswitches) and enable “switchport mode access” on all ports.

3. Place all interfaces in the management VLAN.

4. The interfaces on the management network that connect to theESXi host need to match the speed and duplex settings of the ESXihost physical network interface cards (NICs).

5. Honeywell recommends that unused ports are shutdown andplaced in an unused VLAN.


136


CHAPTER

9 PREPARING AN ESXI HOST

NOTE: : This section is only applicable to the Essentials Platform.

Before you can create PlantCruise virtual machines, you need toprepare the VMware virtualization environment, including configuringphysical and virtual networks, preparing the ESXi hosts, configuringthe vCenter Server, and adding and organizing the ESXi hosts to thevCenter Server.

If you have an existing VMware virtualization environment, you mayonly need to complete some of these tasks. For example, you mayalready have configured the vCenter Server. Ensure that the planningsections of this guide are read and understood before continuing theimplementation.

ESXi host preparation considerations


ManagementClient

The initial configuration of the virtual environment requires aninternet browser to connect to the management host. A workstationor laptop connected to the management network can be used toperform this role. The requirements outlined in the “vSphere WebClient Software Requirements” section of vSphere Upgrade fromVMware outline the minimum requirements for this hardware.This management client should be retained after the configuration ofESXi hosts. It is required for recovery purposes.For more information, see "Configuring your browser" on page 140.

Hardware isinstalled

Verify that the required hardware, such as drives, memory, and NICs,have been installed.

Honeywell recommends that you enable start-up and shutdownbehavior of virtual machines. If you do not specify this behavior, youwill need to manually start each virtual machine on an ESXi host afterthe ESXi host is restarted.

If you are using the Premium Platform, which supports DRS and HA,refer to the Virtualization with Premium Platform guide as HAsupersedes and disables the pre-ESXi host settings. Otherwise, see

137

"Configuring the virtual machine load order on the ESXi host" onpage 158.

Configuring the ESXi host local disk arrayVirtualization Platforms sourced through Honeywell have the localdisk arrays pre-configured. If you need to reinstall, or you sourcedyour platform through another provider, the guidance in this sectionwill be of assistance.

The RAID configuration of a performance production host should beRAID 10 with a hot spare. The RAID configuration of a lower capacityproduction host can be RAID 1.

The following table provides guidance on RAID configuration basedon the usage of the ESXi host:

ESXi host type RAID configuration

Production ESXi host in aSCADA architecture.

RAID 10 with a hot spare.

Production ESXi host in aDCS architecture.

RAID 10 with a hot spare.

Production ESXi host in a DCSarchitecture with low capacity.

RAID 1.1

Management ESXi host. RAID 1.2

Create multiple groups:

l One group for management workloads (whichwill form themanagement datastore).

l One group for staging PlantCruise virtualmachine installations (which will form thestaging datastore).3

Prior to ESXi installation, confirm that the preferred boot device isselected. Refer to the appropriate Dell document for the RAID

1If the hardware is capable then “Production host in a DCS architecture” configurationis permitted2If the hardware is capable then “Production host in a DCS architecture” configurationis permitted3Higher performance host / RAID configuration to be considered

138

Chapter 9 - Preparing an ESXi host

139

controller on the server hardware for instructions to assign apreferred boot device.

Configuring the ESXi host BIOS settingsVirtualization technology abstracts the PlantCruise application fromthe ESXi host hardware so that when virtualized, PlantCruiseinteractions with the BIOS are minimized.

Virtualization Platforms sourced through Honeywell have the optimalBIOS settings pre-configured. If you need to reinstall, or you sourcedyour ESXi host through another provider, the guidance in this sectionwill be of assistance.

Use the following steps to deploy an ESXi host with a BIOS that isoptimized for virtualization. For detailed instructions on how to viewand update the BIOS settings, see the host computer manufacturer'sdocumentation.

1. Use the latest version of the BIOS that is available for yourvirtualization host.

2. Confirm that the following BIOS settings have been set on theESXi host:l Sockets and cores enabled.l 64-bit mode enabled.l (VT) Intel Virtualization Technology enabled (for CPU and for

memory).l Turbo Mode enabled.l Execute Protection feature is enabled (for Intel, eXecute Disable

(XD) enabled).l Hyper Threading enabled.

ATTENTION: You may have to enable this through thevSphere Web Client.

l Node Interleaving disabled.l Unused hardware disabled.l Hardware clock is set to UTC.l Power Management set to Maximum Performance (that is,

disable Power Management).

For additional information related to recommended BIOS settings onan ESXi host, refer to the VMware document, Performance Best


Practices for VMware vSphere® x, where “x” is the vSphere release ofthe ESXi hypervisor.

For additional information relating to ESXi host platformspecifications pertaining to validated BIOS revisions, refer to the HPSVirtualization Specification.

Configuring your browserHoneywell recommends using the vSphere Web Client with thevCenter Server Appliance (VCSA) version 6.5, and that you use thenative web client to connect directly to ESXi servers.

Use a web browser to navigate to the ESXi host or vCenter addressand choose vSphere Web Client (Flash).

Browsers supported are Internet Explorer or Google Chrome.

Handling certificate error warningsIf your vCenter Server uses its own certificate, your browser willdisplay a warning that your connection is insecure or page is notprivate when you attempt to connect. To avoid this warning:

1. From your browser, navigate to your vCenter Server (for example,https://yourvcenterserver.local).a. If you are using Google Chrome, click Advanced, and then click

Proceed....b. If you are using Internet Explorer, click Continue to this website

(not recommended).2. Click Download trusted root CA certificates, and choose where to

store the .zip download.3. In Windows Explorer, expand the downloaded .zip file.4. Locate and double-click the .crt file. Click Open on any security

warnings encountered.5. In the Certificate Viewer, click Install Certificate.6. Select either Current User or Local Machine, depending on who

should be impacted by this change.7. Select Place all certificates in the following store, then click Browse....8. Select Trusted Root Certificate Authorities, then click OK.9. Click Next, then Finish.

10. At the security warning, click Yes, then OK.11. Close the Certificate Viewer.

140


141

For more information, see the VMware Knowledge Base article:https://kb.vmware.com/s/article/2108294.

Using Internet Explorer as your browserThere are some options you need to configure before using InternetExplorer as your default web broswer:

n "Activate Adobe Flash" belown "Enable pop-ups" on the next pagen "Enable animations" on the next pagen "Disable SmartScreen filter" on the next page

Activate Adobe Flash

In Windows 10, Microsoft supplies the required Adobe Flashcomponents as part of the Operating system and they are pre-activated.

In Windows Server 2016, Microsoft also supplies the required AdobeFlash components, but they are not activiated by default. To activateAdobe Flash:

1. From the Server Manager applet, turn off the Internet ExplorerEnhanced Security Configuration option.

2. Ensure that the Operating System has been updated with the latestcumulative updates.

3. From an Administrator command prompt, enter the followingcommand: dism /Online /Add-Package/PackagePath:"C:\Windows\servicing\Packages\Adobe-Flash-For-Windows-Package~31bf3856ad364e35~amd64~~10.0.14393.0.mum",then click Enter.

4. Reboot the machine.5. Run Windows updates again to install the latest Adobe Flash

components. To download the updates manually, go tohttps://portal.msrc.microsoft.com/en-IS/security-guidance.

NOTE: The Adobe Flash updates will not install if AdobeFlash has not first been activated.

If you install the updates and Adobe Flash still does not work,complete these additional steps:


https://kb.vmware.com/s/article/2108294

https://portal.msrc.microsoft.com/en-IS/security-guidance

1. Reboot the machine2. From an Administrator command prompt, enter the following

command: dism /online /remove-package/packagepath:"C:\Windows\servicing\Packages\Adobe-Flash-For-Windows-Package-31bf3856ad364e35-amd64~~10.0.14393.0.mum", then click Enter.

3. Reboot the machine.4. From an Administrator command prompt, enter the following

command: dism /online /add-package/packagepath:"C:\Windows\servicing\Packages\Adobe-Flash-For-Windows-Package-31bf3856ad364e35-amd64~~10.0.14393.0.mum", then click Enter.

5. Reboot the machine.

Enable pop-ups

Internet Explorer will, by default, block pop-ups from appearing. Thisblocks the ability to use remote consoles. To enable pop-ups:

1. From Internet Explorer, choose Tools > Internet Options.2. From the Privacy tab, in the Pop-up Blocker section, choose Settings.3. In the Allowed Sites list, add your domain and/or individual IP

addresses as required.4. Click Close.

Enable animations

1. From Internet Explorer, choose Tools > Internet Options.2. From the Advanced tab, in the Multimedia settings select Play

animations in web pages.3. Click OK.

Disable SmartScreen filter

The SmartScreen Filter can impact performance, particularly for sitesdisconnected from the internet. To disable the SmartScreen filter:

1. From Internet Explorer, choose Tools > Safety > Turn offSmartScreen Filter.

142


143

ATTENTION: This action can help to reduce delays inaccessing the Chassis Management Controller, but shouldonly be completed when the system is not connected to theInternet.

2. Select the option to turn off the SmartScreen Filter.3. Click OK.

Using Google Chrome as your browserBy default, Google Chrome does not allow the required Adobe Flashcomponent to run. When you navigate to the vCenter Server thefollowing message is displayed: To view this page ensure that AdobeFlash Player version 11.5.0 or greater is installed.

To resolve this issue:

1. From Google Chrome's Setting page, click Content Setting, andthen Flash.

2. In the Allow list, add the address of your vCenter Server and theaddresses of any iDRAC instances older than iDRAC9.

Adobe Flash updates typically occur automatically, but if GoogleChrome detects that Adobe Flash is out of date the requiredcomponent will be blocked. Resolve this by updating the details atchrome//settings/help.

You can also go direct to chrome//components, locate Adobe FlashPlayer and click Check for Update.

Installing the ESXi software on a hostVirtualization Platforms sourced through Honeywell have ESXi pre-installed. If you need to reinstall, or you sourced your platformthrough another provider, the guidance in this section will be ofassistance.

This section details how to perform a clean install of the ESXi softwareon the host. If you are upgrading or migrating an existing ESXiinstallation, see “Upgrading virtual infrastructure software”.

ATTENTION: Before you begin the clean install of ESXi, reviewany differences that have been introduced with the newvSphere release. Check for related topics in the "Introduction to


http://chrome//settings/help

http://chrome//components

vSphere Installation and Setup" section of vSphere Installationand Setup.

CAUTION: The root password for each ESXi host should be setduring installation. Each ESXi host should not be connected to atrusted network until you have configured a root password on theESXi host.

Prerequisites

n On the host computer, you have set up:l the disk arrayl the BIOS settings

n Review "Introduction to vSphere Installation and Setup" in vSphereInstallation and Setup. The following guidance is for the InteractiveESXi Installation option.

TIP: PlantCruise virtualization media and documentation isavailable for download from the Honeywell Online Supportweb site http://www.honeywellprocess.com/.

To install an ESXi host

1. Install the ESXi host using the instructions in "Installing ESXiInteractively" in the vSphere Installation and Setup guide.

2. If you plan to create a PlantCruise virtual machine on this ESXihost that requires a USB security device (dongle), Honeywellrecommends using a "USB over IP" device rather than USB postsphysically attached to the ESXi host. No further configuration isrequired during ESXi installation.The USB security devices supplied by Honeywell use USB 2.0.Ensure that the USB ports on the ESXi host are also configured forUSB 2.0.

ATTENTION: Honeywell recommends using a “USB over IP”device to enable the virtual machine to retaincommunication with the license dongle.

144



145

3. In the VMware environment, the physical network adapters on theESXi host are named vmnic n , where n is unique numberidentifying the network adapter. During the installation of the ESXisoftware, network adapters are assigned a vmnic number (startingwith 0) in the order in which the adapters are detected. Physicalnetwork adapters are referenced with the vmnic n names whenconfiguring a vSwitch or viewing the vSwitch configuration.Honeywell recommends that the network adapter cards (single ormultiple port) be assigned the same slots in each ESXi host.The set of virtual network adapters configured for a virtualmachine are referenced as vNIC n where n is a unique numberidentifying the virtual network adapter. This naming convention isfor documentation purposes only.

Configuring the ESXi host network settingsAfter you install the ESXi software on a host, you need to configurethe network settings and the ESXi password.

CAUTION: Since the root password for ESXi is blank after theinitial installation, the ESXi host should only be connected to atrusted network until you have configured a new root passwordon the ESXi host.

After you have configured the IP address of the ESXi host from thedirect console, you can connect to the ESXi host from the vSphereWeb Client from your internet browser. However, to complete thenetwork connectivity configuration of the ESXi host, access from thedirect console is required.

Prerequisites

n You have configured the internet browser on a physical node thatis connected to the management network and can access the ESXiWeb client.

n You have the network details for the ESXi host, such as:

l The IP address, subnet mask, and default gateway of thenetwork interface cards (NICs) for the management network.

l Primary and alternative (secondary) DNS server IP addresses.

l DNS search suffixes.


n Familiarity with, or access to the “Setting Up ESXi” section invSphere Installation and Setup .

To configure the ESXi root password and host IPaddress

1. Connect to the ESXi host from the Direct Console to configure theroot password. Use the instructions in the section "Set thePassword for the Administrator Account".

2. Connect to the ESXi host from the Direct Console or from theinternet browser to configure the ESXi host IP address. Use theinstructions in the "Configuring IP Settings for ESXi" section ofvSphere Installation and Setup

3. Associate the IP address with the default pNIC0 used formanagement network.

To configure the ESXi host DNS settings

1. Connect to the ESXi host from Direct Console.

2. Follow the instructions in the "Configure DNS Settings from theDirect Console" section of vSphere Installation and Setup.

3. From the direct console, select Test Management Network toconfirm that you have network connectivity.

Network configuration for a host in a DCSarchitectureAfter you complete the initial network configuration, you need toconfigure the virtual network for the host.

A virtualized DCS architecture requires ESXi hosts at Level 2 and Level3 to support the production workloads as well as an ESXi host tosupport the management workload.

While the network connectivity of the virtualized productionworkloads is the same as in a physical system (for example, FTE atLevel 2 with network isolation from Level 3), there is an additionalnetwork introduced for the management infrastructure. Thismanagement network may span all ESXi hosts in order to providemanagement access to all elements of the virtualized infrastructure.

146


147

Network configuration for hosts with local storage

Figure 9-1: Example configuration of a management ESXi host withlocal storage

ESXi Management

ESXi Management Host Details for DCS Architectures

vmnic 0

vmnic 1

Domain Controller

vCenter ServerUpdate Manager

vDR

Other (SRM ) vSwitch 0

Production Library Workload

Management B

Management A

vSwitch 2

vSwitch 1

vNIC 1vNIC 0

vNIC 0

vNIC 0

vNIC 0

vNIC 0

Use a single vSwitch with one management port. Use two NICs thatare separately connected to Management A (Level 2.5 A router) andManagement B (Level 2.5 B router).

If you plan to use the management ESXi host to stage the installationof PlantCruise virtual machines, you need to configure the FTE virtualswitches; one for FTE Yellow, and another for FTE Green. Do not adduplinks to the physical FTE network. For more information, see therelated topics.


High capacity hosts

Figure 9-2: Example configurations of high capacity production ESXihost with local storage

FTE Network Yellow



vSwitch 0

FTE Network Green

Experion Server A

ACE

ACE

OPC Server

vmnic 1


Flex Station

Flex Station

Flex Station

vmnic 4

vmnic 5

vNIC 0

vNIC 1

vNIC 0vNIC 1

vNIC 0

vNIC 0

vNIC 1

vNIC 0

vNIC 1

vNIC 0

vNIC 1

vNIC 1

vNIC 0

vmnic 7

vSwitch 2

vSwitch 1

vmnic 6

vmnic 2

vmnic 3

Experion Server A

ACE

ACE

OPC Server

Flex Station

Flex Station

Flex Station

FTE Network Green

FTE Network YellowvSwitch 1

vSwitch 2

vSwitch 3

vSwitch 4

vmnic 4

vmnic 5

vmnic 6

vmnic 7

vNIC 0

vNIC 1

vNIC 0vNIC 1

vNIC 0

vNIC 0

vNIC 1

vNIC 0

vNIC 1

vNIC 0

vNIC 1

vNIC 1

vNIC 0



vSwitch 0 vmnic 1

Management Network Bvmnic 2

vmnic 3

You need a NIC allocation scheme for each production ESXi host. Formore information, see the related topics.

148


149

The high capacity host shown in the first image is configured to usethe on-board multi port NIC to connect the ESXi managementcomponent to the physical management network. The virtual switch,vSwich0, which is created by default as a result of installing ESXi, isused for this connection. A single port from each external multi portNetwork Adapter Card is used to connect Level 2 productionworkload to FTE. In the first example figure, vmnic4 and vmnic6 arethe respective yellow and green FTE connections. Using a single portfrom each of the external multi port Network Adapter cards as shownabove minimizes the scope of loss in the event one of the externalNetwork Adapter Cards should fail. Two additional virtual switches,vSwitch1 and vSwitch2, must be created and configured to enableconnectivity from the production workload to the physical FTEnetwork.

The high capacity host shown in the second image is an example of aproduction host with dual FTE connectivity. In this example, theconnectivity to the management network is identical to that shown inthe first figure. However, the FTE workload is split between 2 separateconnections from the ESXi host to the physical FTE network. Thisconfiguration fully utilizes the ports available on the two externalmulti port Network Adapter Cards. Note that one of the external multiport Network Adapter cards is used for the yellow connection (seevmnic4 and vmnic5 in the second example) while the second externalmulti port Network Adapter card is used for the green connection (seevmnic6 and vmnic7 in the first example). Four virtual switches,vSwitch1, vSwitch2, vSwitch3, and vSwitch4, must be created andconfigured to enable connectivity from the production workload tothe physical FTE network.

ATTENTION: Follow the instructions in the "Setting UpNetworking with vSphere Standard Switches" section ofvSphere Networking, and the tables below to create andconfigure vSwitches. Note that vSwitch0 will already exist as aresult of ESXi installation and host configuration. Refer to the"Port Group Configuration for Virtual Machines" section ofvSphere Networking to create or edit a standard vSwitch with avirtual machine port group. To configure the remainingproperties, such as speed and duplex, refer to "vSphereStandard Switch Properties" in vSphere Networking.

Use the following table to configure the virtual switches in each Level2 production ESXi host that is deployed with single FTE connectivityas shown in the first figure.


Virtualswitch

VMkernelport

Virtualmachine portgroup

UplinkSpeed andduplexvalue

vSwitch0 Managementnetworkname

None vmnic0 (connected tomanagement physicalswitch A)

vmnic1 (connected tomanagement physicalswitch B)

100/Full

vSwitch1 None FTE Yellow 1 vmnic2 (connected to FTEyellow physical switch)

100/Full

vSwitch2 None FTE Green 1 vmnic3 (connected to FTEgreen physical switch)

100/Full

Use the following table to configure the virtual switches in each Level2 production ESXi host that is deployed with dual FTE connectivity asshown in the second figure above.

Virtualswitch

VMkernelport

Virtual machine portgroup

Uplink

Speedandduplexvalue


None vmnic0 (connectedto managementphysical switch A)

vmnic1 (connectedto managementphysical switch B)

100/Full

vSwitch1 None FTE Yellow 1 vmnic2 (connectedto FTE yellowphysical switch)

100/Full

vSwitch2 None FTE Green 1 vmnic4 (connectedto FTE greenphysical switch)

100/Full

vSwitch3 None FTE Yellow 2

Both FTE Yellow 1 andFTE Yellow 2 should beon the same dual NIC.

vmnic3 (connectedto FTE yellowphysical switch)

100/Full

150


151

Virtualswitch

VMkernelport

Virtual machine portgroup

Uplink

Speedandduplexvalue

vSwitch4 None FTE Green 2

Both FTE Green 1 andFTE Green 2 should beon the same dual NIC.

vmnic5 (connectedto FTE greenphysical switch)

100/Full

ATTENTION: The virtual machine port group is created bydefault for the management network as a result of the ESXiinstallation. Honeywell recommends that this port group beremoved from the production ESXi host.

Low capacity hosts

Figure 9-3: Example configuration of an production ESXi host withlocal storage containing low capacity workloads

Console Station vNIC 0

vNIC 1



vSwitch 0

vmnic 1


FTE Network Yellow

FTE Network Green

vmnic 2

vmnic 3

vmnic 5

vSwitch 2

vSwitch 1

vmnic 4

The production host illustrated in the above example uses a NICallocation scheme for a single virtual machine host. For moreinformation, see the related topics.


ATTENTION: Follow the instructions in the "Setting UpNetworking with vSphere Standard Switches" section ofvSphere Networking and the tables below to create andconfigure vSwitches. Note that vSwitch0 will already exist as aresult of ESXi installation and host configuration. Refer to the"Port Group Configuration for Virtual Machines" section ofvSphere Networking to create or edit a standard vSwitch with avirtual machine port group. To configure the remainingproperties, such as speed and duplex, refer to "vSphereStandard Switch Properties" in vSphere Networking.

Use the following table to configure the virtual switches in each Level2 production ESXi host.

Virtualswitch

VMkernelport

Virtualmachine portgroup

UplinkSpeed andduplexvalue


None vmnic0 (connected tomanagement physicalswitch A)

vmnic1 (connected tomanagement physicalswitch B)

100/Full

vSwitch1 None FTE Yellow 1 vmnic2 (connected to FTEyellow physical switch)

100/Full

vSwitch2 None FTE Green 1 vmnic4 (connected to FTEgreen physical switch)

100/Full

ATTENTION: The virtual machine port group is created bydefault for the management network as a result of ESXiinstallation. It is recommended that this port group be removedfrom the production ESXi host.

Network configuration for a host in a SCADAarchitectureAfter you complete the initial network configuration, you need toconfigure the virtual network for the management host.

152


153

Network configuration for hosts with local storage

The minimum three ESXi host network configuration for a SCADAarchitecture using only local ESXi storage. Flex Station andEngineering Station thin clients connect to the production network,while any physical vCenter Client nodes connect to the managementnetwork. The backup storage device is for the backup of virtualmachines on the ESXi hosts.

An ESXi host using local storage and containing only managementworkloads, connects to the management network only.

Figure 9-4: Example configuration of a management ESXi host withlocal storage

Management Network

ESXi Management

Management ESXi Host Details with Local Storage for SCADA Architecture

vmnic 0

vmnic 1

Primary DC vNIC 0

vCenter Server vNIC 0

vDR vNIC 0

Other (SRM ) vNIC 0

vSwitch 0

Configuring NIC teamingConfigure the NIC teaming settings. A NIC team can share the trafficload between the physical and virtual networks among some or all ofthe NIC team members, or provide passive failover in the event of ahardware failure or a network outage.

To configure NIC teaming

1. On the vSphere Web Client (within the Home > Inventories > Hostsand Clusters view), locate the ESXi host. Click on the Configure tab,and then under the Networking group, click Virtual Switches.

2. Select the switch, then click the Manage the physical networkadapters connected to the selected switch icon..


The Manage Physical Network Adapters for <switch> dialog appears.

3. To construct a NIC team (connecting redundant network adaptersto the management network switch), do the following:a. Click on the Add Adapters icon above the Navigation pane.b. Choose an adapter and ensure the Failover order group option is

set to Active Adapters, then click OK..

ATTENTION: If there are already adapters connected tothe vSwitch, the selected adapters must belong to thesame Layer 2 broadcast domain as the existing adapter.

c. Click OK. If a warning All active physical network adaptersassigned to the switch are disconnected appears, click OK.

4. To set the NIC teaming policy exceptions, do the following:a. Select the switch above the table, and click the Edit Settings

icon.b. In the left pane, click Teaming and Failover.c. Set the following policy exceptions:

Policy exception name Value

Load Balancing Route based on the originatingvirtual port

Network FailoverDetection

Link status only

Notify Switches Yes

Failback Yes

d. Click OK.5. Click Close to close the vSwitch Properties dialog box.

Configuring the virtual switch port security settingsTo protect virtual machines from intrusion, the virtual switch portsecurity settings should be modified from the default values.

Prerequisites

n vSphere Networking.

To configure the virtual switch port security settings

154


155

1. Configure the security policy on a virtual switch by following theinstructions in the “Edit Security Policy for a vSphere StandardSwitch” topic in VMware's vSphere Networking GuideUse the following table to identify the policy exception:

Policy exception name Value

Promiscuous Mode Reject

MAC Address Changes Reject

Forged Transmits Reject

For each network attached to your vSwitch, you need to ensure thatthe failover order of the adapters is configured such that there's onlyever one active port.

To override the failover order settings

1. Click the second Edit button to edit the properties of a network onthe vSwitch (not the properties of the vSwitch itself).

2. Select Teaming and Failover.3. Select the Failover order > Override check box.4. Move all but one adapter from the Active adapters list to the

Standby adapters list.5. Repeat these steps for all networks attached to the vSwitch.

Hardening the ESXi host networkTo ensure the ESXi host is hardened against network lockups due toBPDU packets being emitted by VMs, the following setting needs tobe made:

1. From the vSphere Web Client, choose the Home > Hosts andClusters view.

2. Locate and select the ESXi host that requires configuration.3. From the Configure tab, open the System section and click Advanced

System Settings.4. At the top right of the page, click Edit.

The Edit Advanced System Settings dialog box appears.

5. Type Net.BlockGuestBPDU in the Filter field.The Net.BlockGuestBPDU row is displayed in the table.


6. Modify the Value to be 1.7. Click OK.

An event appears in the vSphere Web Client recent tasks status barstating Update options value.

Configuring ESXi host time synchronizationESXi hosts require time synchronization that is accurate whencompared to the time used within the whole infrastructure. Ensurethat you have determined the time source for all ESXi hosts andconfigured the time configuration.

To configure ESXi host time synchronization

1. In the vSphere Web Client, choose the Home > Hosts and Clustersview.

2. Locate and select the ESXi host that requires time configuration.3. Click the Configure tab.4. Under System, click the Time Configuration option.5. At the top right of the page, click Edit.

The Edit Time Configuration dialog box appears.

6. Click the User Network Time Protocol (Enable NTP Client) option.7. Change the NTP Service Startup Policy to Select Start and stop with

host.8. Type the IP address or host name of the NTP time server in the

NTP Servers field, then click Restart, then Yes, and OK.You will see an event in the vSphere Web Client recent tasks statusbar stating Update service activation policy.

Results

Within 15 minutes, the ESXi host time should synchronize with theNTP time server.

Adding and renaming datastoresDuring the installation of an ESXi host, one datastore is formatted asvmfs and named as a default “datastore” or "datastore(1)" format. Thisdatastore name needs to be changed to help identify the ESXi Hostthat the datastore belongs to, and to help identify where a virtual

156


157

machine storage is located in the future.

To assist with identification of storage resources within the virtualinfrastructure, Honeywell recommends the use of a datastore namingconvention. An example convention is:

Name:DatastoreX

Where:

n Name is the ESXi host

n X is the number that is used to identify the datastore

To rename an existing datastore

1. Using the vSphere Web Client, choose the Home > Inventories >Hosts and Clusters view.

2. Locate and select the ESXi host that needs its datastore to berenamed.

3. Click the Configure tab.4. In the left pane, expand Storage and click Datastores.

The existing datastore(s) are listed.

5. Right-click on the name of the datastore that needs to be renamedand select Rename.

To add a new datastore to an ESXi host


2. Locate and select the ESXi host to which you want to add adatastore.

3. Click the Configure tab.4. In the left pane, expand Storage and click Datastores.

The existing datastore(s) are listed.

5. Click Create a new Datastore.The New Datastore dialog appears.

6. Set the Type as VMFS, then click Next.7. Specify a Datastore name, then click Disk/LUN, then click Next.8. For set the VMFS Version as VMFS 5, then click Next..


9. Check the disk layout and confirm that the correct partition andsize are selected.

TIP: The default values are usually adequate unless you needto specify a smaller size than the available disk space..

10. Click Next.11. Review the General and Device and Formatting properties, then click

Finish.The datastore will be formatted into a vmfs datastore ready forvirtual workload to use.

Configuring the virtual machine load order onthe ESXi host

NOTE: This section is only applicable to the Essentials Platform.For Premium Platform users, see the Virtualization with PremiumPlatform Guide for guidance with changing the VM restartpriorities.

Management ESXi hosts require configuration to ensure that themanagement virtual machines are automatically started and started ina specific order.

As part of this configuration, you can also specify a delay for virtualmachines to start up or shut down. A startup delay is recommendedto minimize overburdening the resources of an ESXi host by limitingthe number of simultaneous virtual machine start ups or shut downs.

Honeywell recommends that you enable the startup and shutdownbehavior of all virtual machines. If you do not specify this behavior,you will need to manually start each virtual machine on an ESXi hostafter the ESXi host is restarted.

The recommended start-up order for workloads is:

1. Domain controller

2. vCenter server

3. Other critical management workload.

4. Critical process workloadsAll other workloads can be assigned to Any Order.

To configure the virtual machine load order:

158


159

1. Select the host.2. From the Configure tab, select Virtual Machines > VM

Startup/Shutdown.3. Click Edit.4. Configure the following settings.

Configuration Item Setting

System influence: Automatically start andstop the virtual machines with the system

Selected

Startup Delay 120 seconds

Continue immediately if the VMware toolsstart

Selected

Shutdown delay 120 seconds

Shutdown action Power off

Per VM Overrides: Automatic Startup For process operational workloads,machines should be in the followingorder:

1. Windows domain controller

2. vCenter Server

3. EBR application

4. VMware Site recovery manager

5. Other critical management nodesCritical process workloads

Per VM Overrides: Any Order All remaining workloads

Domain name resolutionESXi hosts need to be manually added to the domain name server(DNS).

The domain name server (DNS) must contain the name of the ESXihost. You must manually add these names to the DNS server, byadding a host 'A’ record to the appropriate Forward Lookup zone.


ATTENTION: The names of virtual machines do not need to bemanually added to the DNS server.

To verify name resolution:

n Verify that vCenter Server can resolve the host name of each ESXihost in the inventory.

n Verify that each ESXi host in the vCenter Server inventory canresolve the name of the management node where vCenter Serveris installed.

160


CHAPTER

10 PREPARING A VCENTER SERVER

On the management ESXi host, you need to deploy the vCenter ServerAppliance (VCSA). The VCSA contains the required softwarecomponents for the administration of virtual machines, ESXi hosts,and the virtualization environment. You need to install a vCenterServer to manage the virtual environment.

vCenter Server preparation considerations


Managementhostworkload

The vCenter Server will run as a virtual machine on the managementhost in the virtual infrastructure. It is important to consider thenumber of virtual machines that will also run on the managementhost so that adequate resources are available. All template creationand storage should also be encapsulated by the management hostso that production workloads are not affected by this type ofoperation. For more information, see the related topic.

Windowsdomaincontroller

A Windows domain must exist before you create the vCenter Server.Ensure that a physical domain controller exists that vCenter cancommunicate with or install a new Windows domain controller onthe management host. The Windows domain controller must:

l Perform the role as a DNS server

l Use reverse lookup zones for all subnets in the infrastructure

l Use DNS HOST A and PTR records to the Forward and Reverselookup Zones for each of the ESXi hosts, using their managementnetwork IP address. For more information about setting up aWindows domain controllers, see the Window Domain andWorkgroups Implementation Guide available from the HoneywellProcess Solutions Support web site.

VirtualMachineReplication

When virtual machine replication is used the vCenter Server requiresextra applications to be installed. Installing these applications duringthe initial installation and configuration phase of the vCenter serverwill prevent possible restarts later when this type of interruption maynot be acceptable. Refer to “Replicating a virtual machine” for anoverview of this functionality and requirements in the vCenter Server.

161


vCenterServer hostaffinity

If you are running a vCenter Server in a vSAN, you can use theaffinity rule to make it easier to restart the vCenter Server in a coldboot situation after the entire cluster has been powered off.

Configuring the vCenter Server with a should rule tying it to the firsthost in a cluster makes it easier to start the VM, as it will normally befound in the first host's register of VMs. If you're using the PremiumPlatform HD, you can use the Host01_VMs_Should group for thispurpose.

Installing a vCenter Server

There are two installation options for vCenter Server:

1. "Installing vCenter Server Appliance 6.5" below2. "Migrating Windows vCenter to vCenter Server Appliance 6.5" on

page 167

Installing vCenter Server Appliance 6.5To set up a vCenter server instance for new virtual systems:

1. Log into a Windows machine connected to the vCenters host’smanagement network.

2. Locate and copy the latest Honeywell-qualified VMware VCSA ISOinstallation disk to the machine, and mount the ISO as a DVDdrive.

3. From the DVD drive, navigate to the vcsa-ui-installer\win32installer\ directory and run the Installer application.The vCenter Server Appliance 6.5 installer wizard appears.

4. Click Install to start the installation, then Next on the Introductorypage.

5. Select the check box to accept the license agreement, then clickNext.

6. From the Deploy Appliance page, select Embedded Platform ServiceController, then click Next.

162

Chapter 10 - Preparing a vCenter Server

163

NOTE: For assistance with a more complicated installationthan this, which is the default, consult the VMwaredocumentation.

7. From the Appliance deployment target page, enter the followingdetails:

Item Description

ESXi host or vCenter Servername

IP address of the ESXi nodethat will host the vCenterServer

HTTPS port Secure HTTP port. Default is443.

User name The ESXi host user name.Default is root.

Password The root password for the ESXihost.

8. Click Next, then click Yes to accept and dismiss the Certificatewarning.

9. From the Set up appliance VM page, supply a Name and Passwordfor the appliance VM, then click Next.

10. From the Select deployment size page, select an appropriateDeployment size and Storage size for your appliance.

TIP: Select a size that enables growth of your system.

11. From the Select datastore page, select the Storage location forthis vCenter Server, then click Next.

NOTE: Leave the Enable Thin Client Disk Mode optionunselected.

12. From the Configure network settings page, enter the followingdetails for the vCenter Server, then click Next:

Item Description

Network


Item Description

IP version

IP assignment

System name Fully-qualified domain name.If not known, leave blank andthe IP address will be used.

IP address IP address of the vCenterServer

Subnet mask or prefix length

Default gateway

DNS servers

13. From the Ready to complete stage 1 page, review the details andclick Finish.The appliance is created and configured on the ESXi host. ClickContinue when the deployment has continued.You are now ready to start Stage 2 of the installation process.

14. From the Introduction page, click Next to Set up vCenter ServerAppliance.

15. From the Appliance page, select the Time synchronization mode, andthe server with which this appliance should synchronize, then clickNext.

16. From the SSO configuration page, enter the local SSO details, thenclick Next.

NOTE: Honeywell recommends using vsphere.local. ThevCenter will be added to a domain later in the process.

17. Click Next to continue past the Customer Experience ImprovementProgram page.

18. From the Ready to complete page, check the details and clickFinish.

19. Read the Warning about not pausing the process, and click OK toproceed.Setup can take some time, as indicated by the progress bar. Whenthe setup is complete, click the address link to open the web client.

20. Click vSphere Web Client (Flash) to configure the appliance.

164


165

21. From the VMware vCenter SIngle Sign-On page, [email protected] as the User Name, and the Passwordas configured during installation.

22. From the vSphere Web Client, choose Home > Administration.23. From the Navigator, click System Configuration to display the System

Configuration panel.24. From System Configuration in the Navigator pane, click Nodes, then

select the vCenter node.25. For the selected node, click the Manage tab, then Networking.26. Change the Hostname to a valid domain host name.27. For the selected node, click Active Directory, then click Join....28. Enter your login details, including the Domain name, User name, and

Password, then click OK.29. For the selected node, choose the Actions menu, then click

Reboot. Provide a reboot reason, then click OK.The reboot will take around 15 minutes to complete. After thattime, refresh the browser and log in again using [email protected] account.

30. From the Navigator, click Administration to display theAdministration panel.

31. From Single Sign-On in the Navigator pane, click Configuration, thenselect the Identity Services tab.

32. Click the Add icon to launch the Add identity service wizard.33. Click Next to accept the Active Directory (Integrated Windows

Authentication) option.34. Check the pre-filled domain details, then click Next, and then

Finish.35. From Administration in the Navigator pane, click Access Control, then

Global Permissions.36. Click the Manage tab to display the current Global Permissions.37. Click the Add icon to display the Select Users/Groups dialog.38. From the list of available Users and Groups, select the required

Domain Access Groups, then click OK.39. Log off and try logging in using login in details for the newly

added domain.


Next steps

n Add hosts to the vCentern Add permissions to the systemn Configure Update Managern Download updatesn Upgrade Hosts to ESXi 6.5n Build and configure virtual machines

166


167

Migrating Windows vCenter to vCenter ServerAppliance 6.5To upgrade an existing Windows-based vCenter 6.0 server to thevCenter Server Appliance:

1. Log into a Windows machine connected to the vCenters host’smanagement network.

2. Locate and copy the latest Honeywell-qualified VMware VCSA ISOinstallation disk to the machine, and mount the ISO as a DVDdrive.

3. Use a copy of the same ISO and mount as a DVD on the WindowsvCenter Server machine.

4. From the DVD drive of the Windows vCenter Server machine,navigate to the migration assistant\ directory and run theVMware-Migration-Assistant application.

5. Enter the [email protected] and domainpasswords as when prompted.

6. Wait until the Waiting for migration to start... messageappears. Do not close the application.

7. From the DVD drive of the client machine, navigate to the vcsa-ui-installer\win32 installer\ directory and run theInstaller application.The vCenter Server Appliance 6.5 installer wizard appears.

8. From the Introduction page, click Migrate, then Next .9. Select the check box to accept the license agreement, then click

Next.10. From the Connect to source server page, enter the IP address of

the vCenter Server and leave the Port value as is pre-filled.11. Enter the SSO User name and Password as configured during

installation ([email protected]), then click Next.12. Click Yes to accept and dismiss the Verity Thumbprint message.13. From the Appliance deployment target page, enter the details of the

ESXi node that will host the vCenter, including the root password,then click Next.

14. From the Select folder page, select the VM folder in whicih youwant to create the appliance VM, then click Next.

15. From the Select computer resource page, select the computer onwhich to deploy the appliance, then click Next.

16. Click Yes to accept and dismiss the certificate warning.


17. From the Set up target appliance VM page, enter a password forthe platform services controller, then click Next.

TIP: Keep a record of this password

18. From the Select deployment size page, select an appropriateDeployment size and Storage size for your appliance.

TIP: Select a size that enables growth of your system.

19. From the Select datastore page, select the Storage location forthis vCenter Server, then click Next.

NOTE: Leave the Enable Thin Client Disk Mode optionunselected.

20. From the Configure network settings page, enter details of atemporary IP address for initial deployment of the vCenter Server,then click Next: After deployment, the appliance will change tousing the current vCenter IP address.

Item Description

Network

Temporary network settings

IP version

IP assignment

Temporary IP address IP address of the vCenterServer

Subnet mask or prefix length

Default gateway

DNS servers

21. From the Ready to complete page, check the details and clickFinish.The appliance is created and configured on the host. ClickContinue when you receive the notification that the deployment iscomplete.You are now ready to start Stage 2 of the migration process.

168


169

22. From the Introduction page, click Next to Migrate Source vCenterServer Appliance for Windows.

23. Address any warnings that are displayed, then click Close.

NOTE: Update Manager and Dell Open Manage warningscan be ignored, but you will need to install a new version ofDell Open Manage after you complete the currentmigration.

24. From the Join AD Domain page, use a domain Administratoraccount to add the vCenter to the Active Directory domain, thenclick Next.

25. From the Select migration data page, select the Configuration,events, tasks, and performance metrics option, then click Next.

26. Click Next to continue past the Customer Experience ImprovementProgram page.

27. From the Ready to complete page, check the details and clickFinish.

28. From the Shutdown Warning dialog, click OK to acknowledge anddismiss the warning.Setup can take some time, as indicated by the progress bar. Whenthe setup is complete, click the address link to open the web client.

NOTE: From vSphere 6.0 onwards, the vSphere Web Clientmust be used for vCenter management tasks.

29. Log off and log into the vCenter using [email protected] account and password.

30. From the vSphere Web Client, choose Home > Administration.31. From Single Sign-On in the Navigator pane, click Configuration, then

select the Identity Services tab.32. Check that the domain is listed, and if not you need to add the

Active Directory Identity Source. For assistance, see step 32 in"Installing vCenter Server Appliance 6.5" on page 162

33. Log off and try logging in using login in details for the newlyadded domain.


Next steps

n Configure Update Managern Download updatesn Upgrade Hosts to ESXi 6.5n Upgrade virtual machines

Organizing VMware inventory objectsYou should identify how to organize inventory objects in vCenterServer.

A logical grouping of inventory objects will help you more easilylocate ESXi hosts and virtual machines within vCenter Server.

You should identify this grouping and then create the requiredinventory objects in vCenter Server, such as datacenters and folders,before adding the ESXi host to vCenter Server.

The "Planning how to organize the vCenter Server" section of this guideprovides more detail and planning considerations on the layout of theinventory. This planning section should be read and fully understoodbefore commencing the organization of inventory objects.

Naming the virtual machine

n Within the inventory objects virtual machines are created. Thesevirtual machines need to be clearly identified. The namingconvention used for virtual machines may include the guest OSname and optionally any other information of use.

n An example Virtual Machine naming convention is Hostname-XXXX,where:

l Hostname is the name of the guest operating system

l XXXX is optional information of significance respective toapplications and system implementation.

n An example Virtual Machine name using this convention is:esxivCenter-Cluster1

170


171

ATTENTION: Virtual Machine names may be used by otherapplications and scripts. Avoid extensive names and specialcharacters.

Adding the ESXi host to the vCenter ServerAdd new ESXi hosts to the vCenter Server.

Prerequisites

n vCenter Server and Host Management.

n The Windows hosts file on vCenter Server must be updated toinclude the ESXi host name resolution (host name or IP address).

To add an ESXi host to vCenter Server

1. Add the ESXi host to the vCenter Server by following theinstructions in the “Add Hosts” topic in the “Organizing YourInventory” section of vCenter Server and Host Management.

Configuring VMware sequential MAC addressallocation

You can enable the VMware Sequential MAC Address Allocation in thevCenter Server to ensure the FTE NIC MAC addresses are correctlyordered. This setting is critical for all systems using FTE and must beconfigured before creating the FTE-enabled VMs.

1. From the vSphere Web Client, navigate to Hosts and Clusters, thenselect the vServer root object from the navigation pane.

2. From the Configure tab, open the Settings section and clickAdvanced Settings.

3. Click Edit.4. Type config.vpxd.macAllocScheme.method in the Name field.5. Type Sequential in the Value field, then click Add.6. Click OK


NOTE: When relying on this feature for FTE, you must create theNICs sequentially to ensure that the MAC addresses are alsosequential. If a NIC has to be deleted and re-added to a VM at anytime, all NICs should be deleted and re-added.

Configuring security in a vCenter ServerCreate vCenter roles and assign them to Experion users or globalgroups, and configure host lockdown mode.

Creating vCenter roles and assigning privilegesvCenter roles are a collection of defined privileges that controlindividual user or group access to particular vSphere objects.

Prerequisites

n You are logged into vSphere Web Client with administratorprivileges.

To create vCenter roles and assign privileges

1. On the vSphere Web Client, choose the Home > Administration >Roles view.

2. Click + to create a Role.The Add New Role dialog box appears.

3. In the Role Name box, type the role name.4. In the Privileges list, select or clear the appropriate privilege check

boxes for this role.5. Click OK to create the role.

Assigning vCenter roles to PlantCruise users or globalgroupsAfter you have defined vCenter roles, assign the role to the requiredPlantCruise users or global groups to associate these accesspermissions to the datacenter and its contents.

172


173

To assign vCenter roles to PlantCruise users or globalgroups

1. On the vSphere Web Client, choose the Home > Inventories > Hostand Clusters view.

2. In the Navigation pane, locate and right-click on the datacenterand choose Add Permission....The Add Permissions dialog box appears.

3. In the Assigned Role list, select the role to be assigned.4. At the bottom of the Users and Groups pane, click Add.

The Select Users/Groups dialog box appears.

5. Select the appropriate Windows domain and then select the useror global group to be assigned the role and click Add for each.

6. Click OK.7. At the bottom of the Assigned Role pane, select the Propagate to

Children check box to propagate the user/role to all hosts andvirtual machines in the datacenter.

8. Click OK.

Configuring host lockdown modeHost lockdown mode ensures that the ESXi host is managed onlythrough vCenter Server.

To configure host lockdown mode using vSphere WebClient

1. On the vSphere Web Client, choose the Home > Hosts and Clustersview.

2. In the Navigation pane, locate and click on the host computer.3. Click the Configure tab.4. Under the System group, click Security Profile.5. In the Lockdown Mode section, click Edit.

The Lockdown mode dialog box appears.

6. Select the Normal check box. This setting permits access to thishost only through vCenter Server or directly on the machine.A confirmation message appears.

7. Click OK.


To configure host lockdown mode using the ESXi hostconsole

1. On the ESXi host console, select Configure Lockdown Mode andpress Enter.

2. Press the spacebar to select Enable Lockdown Mode and pressEnter.

3. Press Enter.

174


175

Creating and managing virtual machinesYou have successfully built your virtual infrastructure, and you arenow ready to create and deploy virtual machines to your virtualenvironment.

If you plan to create an Experion virtual machine on an ESXi host thatrequires a USB security device (dongle), see the vSphere VirtualMachine Administration Guide for assistance.

For guidance creating, deploying, and managing virtual machines, seethe Software Installation User’s Guide.


176


CHAPTER

11 ADMINISTERING THE VIRTUALIZATIONENVIRONMENT

In this section:

Replicating a virtual machine using vSphere Replication 178

Restoring and recovering a replicated virtual machine 181

Patching or upgrading the VMware environment 185

Maintaining hardware devices in a virtualization environment 204

Monitoring the virtualization environment 206

177

Replicating a virtual machine using vSphereReplication

For a brief overview of vSphere Replication refer to section “ConfigureReplication for Multiple Virtual Machines to vCenter Server” in theVMware vSphere Replication Administration document.

The following topics describe how to configure the replication ofvirtual machines using VMware vSphere Replication appliance that isqualified for use with Experion Virtualization from VMware vSpherever 5.5, 6.0, and later. Further details of this appliance can be found inthe section “Replicating Virtual Machines” in the VMware vSphereReplication Administration document.

Most nodes that may be found in an Experion system do not includethe inherent redundancy available for the Experion Server. In the caseof a critical non-redundant node, for example an ACE, vSphereReplication can be used to provide a continuously updated replicationof its virtual machine. In the event of failure of the ACE host’shardware this replicated virtual machine can be powered on and runin its place. This will allow for the failed virtual machine to be replacedby the replicated virtual machine while hardware repairs orreplacement is undertaken.

Preparing the vCenter Server for virtual machinereplication using vSphere ReplicationInstall vSphere Replication appliance and configure for single site,single vCenter Server usage.

Prerequisites

n Verify that the management host and management datastore haveenough spare capacity to accommodate the vSphere Appliance.

n Download and install the Client Integration Plug-in from the loginpage of the vSphere Web Client. If the plug-in is already installed,the link does not appear.

n In the vSphere Web Client, select the vCenter Server instance onwhich you are deploying vSphere Replication, click Manage >Settings > Advanced Settings and verify that the VirtualCenter.FQDNvalue is set to a fully-qualified domain name or a literal address.

n Allocate an IP address on the management subnet for the vSphereReplication appliance. Create a DNS record (forward and reverse)

178

Chapter 11 - Administering the virtualization environment

179

for the appliance.

n Download the vSphere Replication zip file to a network locationthat is accessible from the Web Client. Unzip the file.

To deploy the vSphere Replication appliance:

1. Follow the instructions in the section “Deploy the vSphereReplication Virtual Appliance” in the VMware vSphere ReplicationAdministration document.

2. Use the following settings when requested:

l Set the Number of vCPUs to 4.

l For the destination datastore select the management datastorechosen in the pre-requisites at the start of this procedure.

l Set the disk format of the destination datastore thickprovisioning, if applicable.

l From the list of available networks select the managementnetwork.

l Set the network protocol to IPv4

l Select Static-Manual IP addressing and set the IP addressdetails as determined in the pre-requisites at the start of thisprocedure. Ensure you record this password.

l Choose to use embedded database.After clicking Finish the vSphere Replication appliance is deployed.

3. Log out of the vSphere Web Client and close the browser.4. Re-start the vSphere Web Client, log in and view the Home page.

If the vSphere Replication appliance was successful there will be avSphere Replication icon visible.

5. Click vSphere Replication

Ensure the vCenter Server on which the Sphere Replicationapplication was deployed is listed and that vSphere Replication isEnabled.

Preparing the virtual machine for replication usingvSphere ReplicationConfigure virtual machine to replicate to datastore within the singlevCenterServer deployment of vSphere Replication appliance.


Prerequisites

n Replication planning has occurred including:

l Identifying the virtual machines to be replicated (source),

l Identifying the host & datastore to which to replicate (target),

l Ensuring target datastore has sufficient space to accommodatereplicated virtual machine.

n The vSphere Replication appliance has been successfully installedand configured for use with single vCenter Server per the section“Preparing the vCenter Server for virtual machine replicationusing vSphere Replication”.

n Virtual machines must powered on to begin replication.

To configure virtual machines for replication:

1. On the vSphere Web Client Home page, click vSphere Replication.2. Select the vCenter Server in the left pane and double-click Virtual

Machines.On the Related Objects tab, the Virtual Machines tab lists thevirtual machines.

3. Select the virtual machine to be replicated. If multiple virtualmachines are to be replicated use the Ctrl or Shift keys to selectthem.

4. Right-click the virtual machines and select All vSphere ReplicationActions > Configure replication.The Configure Replication dialog appears at the Validation phase.The virtual machines pass a validation check before the dialog cancontinue to the Replication type phase.

5. Click Next

6. At the Replication type phase click the Replicate to a vCenter Serveroption button and click Next.

7. At the Target site phase select the vCenter Server and click Next.8. At the Replication server phase click the vSphere Replication

server option and click the listed replication server.9. At the Target location phase click None in the VM Storage Policy

list. Choose a data store based on the following criteria:

l If you replicate between two hosts with local storage, choosethe datastore of the destination host,

180


181

l If you replicate between two Servers within the same chassis,then select a different datastore than that of the source virtualmachine. For example, if source uses Production1, useProduction2 for target,

l If you replicate between two Servers in different chassis, thenselect any datastore on target chassis.

Select the Advanced disk configuration check box and click Next.

10. At the Hard disk 1 phase click Same format as source in the Selectvirtual disk format list and click Next.

11. At the Replication options phase click MS Shadow Copy Services(VSS) in the Quiescing method list and click Next.

12. At the Recovery setting phase Honeywell recommends thefollowing settings:

l Recovery Point Objective (RPO) = 12 hr,

l Point in time instances check box is select,

13. At the Ready to complete phase Review the settings and clickFinish

vSphere Replication starts an initial full synchronization of theselected virtual machines to the designated target datastore.

14. To monitor a virtual machine’s replication status click the Monitortab in the vSphere Web Client, click Incoming Replications and clickthe required virtual machine. In the Replication Details pane viewthe status of the initial sync and subsequent replications. If theinitial full sync is in progress use the manual refresh icon toupdate its progress.

Restoring and recovering a replicated virtualmachine

In the event of an ESXi Host failure the clone of any replicated virtualmachine can be powered on and run in place of the original. This willallow for the functionality of the original virtual machine to bereplaced by the clone while hardware repairs or replacement isundertaken. The following failure scenarios outline where replicatedvirtual machines can be used:

n Unrecoverable failure of ESXi Host Hardware where noreplacement is available within 1 hour


n Failure of ESXi Host hardware where recovery will take greaterthan 1 hour.

It is important to determine the length of time that the ESXi host willbe in a failed state. If the downtime is estimated to be more than 1hour and a replacement ESXi host is not immediately available, thereplicated virtual machine(s) need to be used to replace the original.

Starting replicated machines

1. On the vSphere Web Client Home page, click vSphere Replicationthen click the Monitor tab.

2. Click Incoming Replications, right-click the virtual machine to berecovered then click Recovery from the drop-down list

3. Follow the procedure “Recover Virtual Machine by Using vSphereReplication” in VMware vSphere Replication Administration.Honeywell recommends using the following options:

l Recovery Options: click the Use latest available data option.

l Folder: Browse to the folder in the inventory to where you wantthe selected virtual machine to be recovered.

l Resource: Click the host within the selected inventory folder towhere you want the selected virtual machine to be recovered.

l Ready to complete: Select the Power on the virtual machine afterrecovery checkbox.

4. Verify that the virtual machine’s status changes to Recovered andthat virtual machine appears in the inventory of the target host.

5. Connect virtual network adapters to the correct network. Use thevirtual machine settings to connect the virtual network of thevirtual machine. Re-join the process control domain to re-establish trust relationship. Restart the Windows OS on the virtualmachine.

6. If the virtual machine is an ACE, restore the latest checkpoint fromControl Builder on a PlantCruise engineering node. Start thecontrol strategies as required.The system is now running in a “replicated” state, where the ESXiHost on which the original virtual machine was running has failedand the replicated virtual machine is now running on a differentESXi host. In a PlantCruise system this model would typically beused for ACE virtual machines. It is important to ensure that theESXi hosts that run the replicated ACE have been provisioned withsufficient resources to run the “replicated” workloads.

182


183

All PlantCruise workloads that are inherently redundant (e.g.Experion Server) that were running on the failed ESXi host willalso be running in a degraded state; they should be synchronizedto restore full redundancy.

ATTENTION: To ensure that any additional hardware failuresdo not affect the process control system Honeywellrecommends that the system only be run in the “replicated”state for as short a time as possible.

Reconnecting a repaired ESXi host

1. Repair the failed ESXi Host.2. Power-on the ESXi Host.3. Using the vSphere Web Client, connect to the vCenter server.4. Allow all virtual machines to automatically start on the repaired

ESXi host.5. Ensure that all virtual machine with a current running replica have

started successfully and that the virtual network to each isdisconnected.

6. Ensure that all virtual machines that do not have a running“replicated” clone are operating normally. This includessynchronizing Experion servers. For each replicated virtualmachine:a. Power off the running replica virtual machine.b. Use the virtual machine settings to connect the virtual network

of the original virtual machine.c. Re-join the process control domain from the original virtual

machine to re-establish a trust relationship and then restart.

ATTENTION: Multiple checkpoint restore requests from thesame engineering station occur one at a time and may takelonger than expected. Requesting one checkpoint restorefrom multiple engineering stations may speed up therestore process.

The PlantCruise System should now be running normally as beforethe ESXi Host failure.


Reconnecting a replaced ESXi host

1. When the failed ESXi host is replaced or the repair is completedbut disk data is lost, you will need to:a. Perform a clean installation of the ESXi hypervisor on the ESXi

hostb. Restore the original virtual machine workload from the latest

vDR backup.2. Replace ESXi host hardware.3. Install ESXi software and configure ESXi. Refer to the “Preparing a

production ESXi host” section in the Experion VirtualizationPlanning and Implementation Guide.

4. Reconnect the ESXi host in vCenter. This will requireacknowledgement of a new certificate for the ESXi host.

5. Restore all virtual machines to the replaced ESXi host using thelatest vDR backup.

6. Start all virtual machines that do not have a running “replicated”clone and ensure that each operates normally. This includesestablishing domain trusts and synchronizing Experion servers.The PlantCruise System should now be running normally as beforethe ESXi Host failure.

7. For each replicated virtual machine:a. Power off the running clone virtual machine.b. Power on the restored original virtual machine.c. Use the virtual machine settings to connect the virtual network

of the virtual machine.d. Re-join the process control domain from the original virtual

machine to re-establish a trust relationship and then restart.The PlantCruise System should now be running normally as beforethe ESXi Host failure.

Failback of Virtual machines

Refer to the procedure “Failback of Virtual Machines in vSphereReplication” in VMware vSphere Replication Administration.

1. Ensure that the original source virtual machine is in the poweredoff state then unregister it from the inventory of the source siteusing the Remove from Inventory command.

2. Configure replication in the reverse direction; i.e. from the originaltarget host/datastore to the original source host/datastore.

184


185

3. Power off the original target virtual machine as per the section“Restarting the virtual machine

ATTENTION: Ensure that all controls have first been placedin a safe condition before performing this step.

4. Recover the original source as per the section “Starting replicatedmachines”.If you recover to the same location as the original source you clickselect Yes to overwrite the virtual machine’s .vmx file.

Patching or upgrading the VMwareenvironment

The virtual environment is made up of the following componentsvCenter Server, ESXi host, virtual machine hardware, VMware Tools,and VMware Data Recovery (vDR).

Patching or upgrading can affect one or all of these components. It isimportant to update each component in the proper order to maintaincommunication and to ensure that data is not lost. Detailedinstructions for performing a upgrade is provided in VMware upgrade.Patches typically apply to individual components. However, ESXi hostpatches can affect a virtual machine’s virtual hardware and/orVMware Tools. The “Remediating vSphere objects” section inInstalling and Administering VMware vSphere Update Managerrecommends upgrading VMware Tools before upgrading virtualhardware.

Honeywell recommends using vSphere Update Manager toadminister and automate the application of patches and upgrades tothe VMware environment, when possible.

Update Manager Client has two views available through vSphere WebClient. The Administration view and the Compliance view. For moreinformation about how to access these view, see the “Update ManagerClient Overview” section in Installing and Administering VMwarevSphere Update Manager.

The Update Manager Client Administration view provides the interfacefor configuring the Update Manager, viewing Update Manager events,creating/modifying baselines and baseline groups, for importingpatches/updates into the patch repository, and for importingupgrades into the upgrade release repository. Using baselines and


baseline groups. you can organize patches from the patch repositoryinto a Honeywell approved set for deployment/remediation.

The Update Manager Compliance view provides the interface forassociating baselines (the desired set of patches) and baseline groupswith target objects, such as hosts, virtual machines, virtual appliances,or containers. A container has one or more hosts, virtualmachines/appliances. The Compliance view provides the interface forscanning the target objects to determine if they contain the patchesdefined in the attached baseline or baseline group.

Target objects containing the patches are considered to be incompliance with the baseline or baseline group. The compliancestatus is also reported in the compliance view. A status of incompliance is green, while a status of non-compliance is red. If theobject is not in compliance with the attached baseline or baselinegroup the Compliance view also provides the interface fordeploying/remediating the patches to the target object. If the target isan ESXi host, the compliance view provides an additional optionalinterface for staging (copying) the patch to the target host inpreparation for remediation. The compliance view provides theinterface for initiating the deployment/remediation of the patch to thetarget object.

Details on these topics can be found in Installing and AdministeringVMware vSphere Update Manager.

ATTENTION: If the vCenter Server resides on the local storageof an ESXi host, Update Manager cannot be used todeploy/remediate patches to that ESXI host. An ESXi must beput into maintenance mode to remediate patches. Maintenancemode requires all local storage virtual machines to beshutdown. A manual procedure is used to deploy/remediatepatches or upgrades to this ESXi host. Alternatively, the vCenterServer may be moved to another host temporarily to managethe patch process.

Prerequisites

n VMware vSphere Update Manager Installation and AdministrationGuide, to understand the patching process.

n vSphere Command-Line Interface Installation and Scripting Guide,for installing the Command-Line Interface (CLI).

n vSphere Upgrade

186


187

n A computer that has access to the Internet for downloadingupdates and patches.

n Update Manager is installed on the vCenter Server.

n You must have the Remediate to Apply Patches, Extensions, andUpgrades privileges associated with your login to remediatevSphere objects.

n You know the vSphere product edition and version that is installed.

n You have a predefined procedure defining the specific order forshutting down and restarting virtual machines, virtual appliances,and ESXi hosts to support a running process.

To patch or upgrade the VMware environment

1. Use the Update Manager air-gap deployment model but do notuse the Update Manager Download Service to retrieve updatesfrom VMware. Honeywell evaluates VMware updates and posts alist of qualified and approved updates on the Honeywell ProcessSolutions web site.

2. Download the approved updates directly from VMware downloadsite. See the “Update Manager Deployment Models” section inInstalling and Administering VMware vSphere Update Manager forguidance on how to set up the air-gap deployment model.

3. Use vCenter Update Manager to:

a. Import patches or upgrades.

b. Manage baselines and baseline groups to contain the desiredpatches.

c. Associate baselines and baseline groups to target objects.

d. Scan objects for compliance with the baseline.

e. Apply patches to ESXi hosts.

f. Upgrade ESXi host to new releases.

g. Upgrade VMware Tools within virtual machines.

h. Upgrade virtual machine hardware versions.

i. Upgrade virtual appliances.

j. Scan objects to verify target objects are in compliance with theattached baselines.


4. Use a manual procedure to apply patches to ESXi hosts thatcontain Update Manager and vCenter Server on local storage.

5. Use Honeywell documented procedures to:

a. Apply PlantCruise patches to virtual machines.

b. Apply Microsoft patches or security updates to virtualmachines.

Downloading the patches or upgradesYou need to identify and evaluate the VMware updates to bedownloaded from the Honeywell Process Solutions website.

Prerequisites

n You need to complete this task on a computer with access to theInternet.

n You need a Honeywell Online Support account.

To identify the required patches from Online Support

1. Logon to the Honeywell Online Support web sitehttp://www.honeywellprocess.com/.

2. Click the Support tab.3. Type VMware Update in the Search field, and press ENTER.

The Honeywell-qualified VMware updates are listed.

4. Review the updates and decide which ones are appropriate foryour environment.

To download patches from VMware

1. The VMware patch release ID on the VMware update summary pageis a link to the consolidated patch .zip file.

2. Click the link and follow the instructions to download theconsolidated patch package.

3. Copy or move the consolidated patch .zip file to a location ormedia that can be accessed by the Management Client.

188



189

Importing the patches or upgrades into UpdateManagerOnce you have downloaded the required patches or upgrades fromthe VMware web site, you need to load these patches or upgrades intoUpdate Manager.

Prerequisites

n VMware vSphere Update Manager Installation and AdministrationGuide

n The patch must be in the zip file format.

n You import patches and upgrades from the Update ManagerClient Admin view.

Patches or updates

1. To import patches or updates, see the VMware vSphere UpdateManager Installation and Administration Guide.

ATTENTION: Patches can also be imported using the ImportPatches link in the Patch Repository push button on theManage tab.

Host upgrades

1. To import ESXi host upgrades, see VMware vSphere UpdateManager Installation and Administration Guide.

ATTENTION: It is important to ensure that a host upgrade isonly performed within the context of a full system upgrade.If an ESXi host is upgraded to a version or releasenewer than the vCenter Server, the ESXi host will notconnect to vCenter Server.

About baseline and compliance viewsUpdate Manager Client has two views available through vSphere WebClient. The Administration view and the Compliance view. For moreinformation on how to access these views, see the VMware vSphereUpdate Manager Installation and Administration Guide.


Administration view

The Update Manager Client Administration view provides an interfacefor creating editing or deleting baselines and baseline groups. Usingbaselines and baseline groups you can organize patches from thepatch repository into the Honeywell approved set fordeployment/remediation. Update Manager also comes with built-in(default) baselines which automatically include updates downloadedfrom VMware.

However, the default baselines cannot be edited. To exclude one ormore of the downloaded patches requires you to create a newbaseline. For information about the default baselines and how tocreate and manage baselines and baseline groups see the VMwarevSphere Update Manager Installation and Administration Guide.

To review the patches assigned to a baseline:

1. From the Admin view in the Update Manager Client, click theManage tab.

2. Click Hosts Baselines or VMs/VAs Baselines.3. Select the number or name in the content column in the list of

baselines.A list of assigned patches appears.

Baselines and baseline groups are used to define a set of patches,updates, or upgrades. Baseline groups allow patch and upgradebaselines to be remediated together.

Compliance view

The Update Manager Client Compliance view allows a user to specifythe hosts, virtual machines, and virtual appliances that will beevaluated against a baseline or baseline group for compliance. Thecompliance view provides a status of the compliance scan/evaluation.A non-compliance status identifies the objects needing to beupdated/remediated with the attached baseline. The compliance viewalso provides the interface for a user to initiate the remediationprocess for the object.

The Update Manager Client Compliance view provides an interfacefor:

n Attaching baselines or baseline groups to target objects or targetcontainers. For more information, see the VMware vSphereUpdate Manager Installation and Administration Guide.

190


191

n Scanning to determine if target objects are in compliance(contains all the patches) defined in the attached baselines orbaseline group. For more information, see the VMware vSphereUpdate Manager Installation and Administration Guide.

n Staging copies of the patch to the target objects in preparation forremediation. Doing this can reduce the time an ESXi host needsto be in maintenance mode. For more information, see theVMware vSphere Update Manager Installation and AdministrationGuide.

n Remediating baselines or baseline groups to target objects. Formore information, see the VMware vSphere Update ManagerInstallation and Administration Guide.

ATTENTION: Honeywell recommends initiating remediationon individual objects rather than initiating remediation onobject container. This is so the order of shutting down andrestarting virtual machines and ESXi hosts is executed in acontrolled and determinate way.

Remediating patches and upgradesHoneywell recommends using Update Manager for determiningpatch and upgrade compliance and for remediation of all hosts,virtual machines and virtual appliances when possible.

A manual procedure is required to remediate an ESXi host withpatches and upgrades if that ESXi host has vCenter Server or UpdateManager hosted on its local storage. This procedure requires theCommand-Line Interface (CLI) to be installed on the client node.

Additional consideration is required when remediating an ESXihost that has virtual machines or virtual appliances running on itslocal storage. The hosted virtual machines and virtual appliances mustbe shutdown before the ESXi host can be put into maintenance mode. The virtual machines should be shutdown in a pre-determined orderto minimize the effect on the running process. You should checkpointACE nodes prior to shutting down.

ATTENTION: Honeywell also recommends user have a pre-defined shutdown and restart order for ESXi hosts, virtualmachines, and virtual appliances to minimize the effect on therunning process.


The Update Manager Client Compliance view provides access to theinterface required for performing the procedures in this topic. Formore information on how to access this view, see the “ScanningvSphere Objects and Viewing Scan Results” topic in the VMwarevSphere Update Manager Installation and Administration Guide.

Prerequisites

n User defined procedure defining the order for shutting down andrestarting ESXi host and any virtual machines or virtual applianceshosted on their local storage.

n VMware vSphere Update Manager Installation and AdministrationGuide to understand the patching process.

n vSphere Command-Line Interface Installation and Scripting Guidefor installing the Command-Line Interface (CLI).

n vSphere Command-Line Interface (CLI) installed on the clientnode performing manual patching and upgrade of ESXi host.

n Baselines and baseline groups are created or identified thatcontain the desired patch or upgrade to be remediated.

n Baselines and baseline groups are attached to the target object.

n Remediating patches and upgrades together requires that thepatch and upgrade baselines be added to a baseline group.

Preparing for remediation

1. Attach baseline groups to ESXi host, virtual machines, and virtualappliances.Baselines or baseline groups can be attached to individual targetobjects or to their container. For more information, see the“Attaching Baselines and Baseline Groups to vSphere Objects”topic in the VMware vSphere Update Manager Installation andAdministration Guide.

2. Scan for compliance.Manually initiate a scan on individual target objects or all targetobjects within a container. For more information, see the“Scanning vSphere Object and Viewing Scan Results” in theVMware vSphere Update Manager Installation and AdministrationGuide.

192


193

3. Stage patches to an ESXi host.Staging is only available for ESXi hosts and is optional. Stagingcopies patches defined in the attached baseline to the target ESXihost while it is still operational reducing the amount of timerequired in maintenance mode. If the ESXi host requires manualremediation, staging has no value. For more information, see the“Stage Patches and Extensions to ESX/ESXi Hosts” topic in theVMware vSphere Update Manager Installation and AdministrationGuide.

4. Repeat the previous steps on all ESXi hosts prior to remediation.

Remediating target objects using Update Manager

Prerequisites

n You have completed the preparation for remediation.

Choose the update type and follow the required procedures:

Type Action

For Hostpatchesonly

Complete the steps in the “Remediate Hosts AgainstPatch or Extension Baseline” topic in the VMwarevSphere Update Manager Installation andAdministration Guide.

For Hostupgradesonly

Complete the steps in the “Remediate Hosts Againstan Upgrade Baseline” topic in the VMware vSphereUpdate Manager Installation and AdministrationGuide.

For Hostpatchesandupgrades

Complete the steps in the “Remediate Hosts AgainstBaseline Groups” topic in the VMware vSphere UpdateManager Installation and Administration Guide.

For VirtualMachinesandAppliances

Complete the steps in the “Remediating VirtualMachines and Virtual Appliances” topic in the VMwarevSphere Update Manager Installation andAdministration Guide.

ViewUpdateManagerEvents

Complete the steps in the “View Update ManagerEvents” topic in the VMware vSphere Update ManagerInstallation and Administration Guide.


Type Action

All types Complete a scan to verify that the remediation wassuccessful and that the target object is now incompliance with the attached baseline.

ATTENTION: When using Update Manager for remediation,Honeywell recommends:

l Using user-initiated and immediate remediation so apredetermined order can be followed for shutting down andrestarting ESXi hosts, virtual machines and virtualappliances.

l Remediating one target object at a time to the attachedbaselines or baseline group.

Use the following input in the Host Remediationwizard:

Entry Recommended value

UniqueName

Patch ID DDMMYYYY, Update ID DDMMYYYY,Upgrade ID DDMMYYYY, or Patch ID- Upgrade IDDDMMYYYY

Description Describe the purpose of the patch, update, orupgrade

Schedule Immediately

FailureResponse

Fail Task

Clusteroptions

Disable if option provided

Use the following input in the Virtual MachineRemediation wizard:


Name Upgrade ID DDMMYYYY

Description Describe the purpose of theupgrade

194


195


Schedule Immediately

Snapshot Select

When to delete Don't delete snapshot

Snapshot Name Update ID DDMMYYYY

Snapshot Description Describe the purpose of the upgrade

Snapshot memory of theVM

No

Remediating target objects manually

Prerequisites

n You have completed the preparation for remediation.

To disable host lockdown mode

1. On the vSphere Web Client, choose the Home > Inventories > Hostsand Clusters view.

2. Locate and click on the ESXi host computer.3. Click the Configure tab.4. Under the System group, click Security Profile.5. Scroll down to Lockdown Mode, and click Edit.

The Configure Host Lockdown Mode dialog box appears.

6. Select the Disabled option.A confirmation message dialog box appears.

7. Click OK.

To put the ESXi host into maintenance mode

1. Using vSphere Web Client from a client node with CLI installed,connect to the ESXi host directly.

2. Shutdown all virtual machines and put the ESXi host inmaintenance mode.


To prepare using the VMware CLI

1. Copy the patch or upgrade .zip package to the client node.Make sure there are no spaces in the path to the patch or upgrade.zip file on the client node.

2. Open a Windows Command Prompt window with administratorprivileges.

3. Open the folder where the vCLI is installed.For example:cd c:\Program Files\vmware\vmware vsphere CLI\bin

To verify the host identity and confirm that it is inmaintenance mode

1. Note: the following commands require the double dash (--)2. Type the following command:

vicfg-hostops.pl --server <ip_address> --operation info

where:<ip_address> is the IP address of the ESXi host.

ATTENTION: The vicfg-hostops.pl command requires thedouble dash (--) for each of the options.

3. Type the username.4. Type the password.

Here is an example of the output. Note the status of MaintenanceMode.Host Name : ESX5-R710.virtlab.localManufacturer : Dell Inc.Model : PowerEdge R710Processor Type : Intel(R) Xeon(R) CPU E5530 @2.40GHzCPU Cores : 8 CPUs x 2393 GHzMemory Capacity : 49141.6640625 MBVMotion Enabled : noIn Maintenance Mode : YesLast Boot Time : 2011-06-24T17:22:02.593417Z

196


197

To verify what is currently installed

1. Type the following command:esxcli--server=<ip_address> software vib list



A list of installed bundles is displayed.

To upload the update bundle to the ESXi host

1. Using vSphere Web Client, select the ESXi Host that requiresupdates. Browse a datastore on the ESXi Host.

2. Create a new folder in the root of the datastore called "updates"3. Navigate into the new updates folder and upload the vmware

update bundle (.zip file).The location of the update bundle is now /vmfs/volumes/<ESXidatastore name>/updates/<update bundle name>

where:

l <ESXi datastore name> is the name of the datastore where theupdate bundle was uploaded

l <update bundle name> is the name of the update bundle (.zip file)

To verify the contents of a .zip package

1. Type the following command:esxcli--server=<ip_address> software sources vib list -d <package_path>

where:

l <ip_address> is the IP address of the ESXi host.

l <package_path> is the path and file name of the .zip package. Forexample, /vmfs/volumes/ESX_Datastore?updates/ESXi500–201303001.zip.


A list of included bundles is displayed. The status of each updatewill be Installed, Update, or New.


To install the required updates

1. Type the following command:esxcli --server=<ip_address> software vib update -d <package_path>

where:

l <ip_address> is the IP address of the ESXi host.

l <package_path> is the path and file name of the .zip package. Forexample, /vmfs/volumes/ESXi_Datastore/updates/ESXi500-201303001.zip.

2. Type the username.3. Type the password. All applicable bundles that are included on the

ESXi Host will be updated. No new bundles will be added.Here is an example of the output, which will list all updated VIBs.Installation result:Message: the update completed successfully, but thesystem needs to be rebooted for the changes to beeffective VIBs Installed: ...

To restart the ESXi host

1. Type the following command:vicfg-hostops.pl --server <ip_address> --operation reboot



The ESXi host restarts. Several errors may be reported afterrunning this command. However, the ESXi host does reboot.

To verify that the specified bulletins were installed

1. Type the following command:esxcli --server=<ip_address> software vib list



A list of installed bundles will be displayed ensure that the install

198


199

date of the updates that were updated reflects the current date.

To restore the management workload back to service

1. From the browser on the client node, connect directly to the ESXihost.

2. Take the ESXi host out of maintenance mode.3. Power on the domain controller virtual machine and allow it time

to start up.4. Power on the vCenter Server virtual machine and allow it time to

start up.5. Start the remaining virtual machines and/or virtual appliances.

To verify that the ESXi host is in compliance

1. Using the vSphere Web Client, connect to the vCenter Server.2. Go to the Home > Inventories > Hosts and Clusters view, then locate

and select the ESXi host computer.3. Click the Update Manager tab.4. Click Scan for updates to scan this ESXi host.5. Select the Patches and Extensions and Upgrades options, then click

OK.The ESXi host should now be compliant.

To enable host lockdown mode using vSphere WebClient


2. In the Navigtaion pane, locate and click on the ESXi hostcomputer.

3. Click the Configure tab.4. Under the System group, click Security Profile.5. Scroll down to Lockdown Mode, and click Edit.

The Lockdown Mode dialog box appears.

6. Select Normal as the Lockdown Mode. This permits access to thishost only through vCenter Server or directly at the machine.A confirmation message dialog box appears.


7. Click OK.

Next steps

n If applying a patch then patching of the host is complete.

n If upgrading then go to the next phase/step of the upgradeprocess.

Upgrading Virtual Hardware and VMware toolsversionsUse the following steps to upgrade virtual machines to use the latestvirtual hardware version and/or VMware Tools with Update Manager.Note that the upgrade requires one or more re-boots of the virtualmachine in order to complete the upgrade.

NOTE: Ensure that the virtual hardware for all virtual machinescomplies with the qualified virtual hardware versions as definedin the HPS Virtualization Specification.

Use this procedure if you are upgrading virtual machines that werecreated on a vSphere release that is older than the current release.This is optional. The current vSphere release will support virtualmachines deployed with older Virtual Hardware or VMware toolsversions. Refer to VMware compatibility guides to determine whichVirtual Hardware or VMware tools versions are supported with eachvSphere release

ATTENTION: Whenever possible, plan to have the same virtualhardware revision in use on a given ESXi host. If you aredeploying virtual machines on a single host with differentvirtual hardware versions you may experience unpredictableperformance behavior.

To upgrade Virtual Hardware or VMware Toolsversions on virtual machines

1. From the vSphere Web Client (Flash) that is connected to vCenterServer, select VMs and Templates Inventory.

2. Power on any virtual machines that are not running by right-clicking on the machine name and clicking Power.

200


201

3. From the navigation pane of the VMs and Templates Inventory view,right-click on your datacenter and click New Folder. Name the newfolder Upgrades.

4. Drag and drop the virtual machines that are to be upgraded intothe new Upgrades folder.

5. Create baselines for your virtual machines:a. From the navigation pane of the VMs and Templates Inventory

page, select the Upgrades folder, and click the Update Managertab.

b. Click Attach Baseline. The Attach Baseline or Baseline Group dialogbox appears.

c. Select the VMware Tools Upgrade to Match Host (Predefined) andVM Hardware Upgrade to Match Host (Predefined) check boxes.

d. Click OK.e. Check that baselines now exist for all your virtual machines.

6. Scan your virtual machines against the baselines:a. Click Scan for Updates.... The Confirm Scan dialog box appears.b. Select the VM Hardware upgrades and VMware Tools upgrades

check boxes.c. Clear the Virtual appliance upgrades check box, then click OK.d. When the scan finishes, check that your virtual machines are

either 0% compliant or n% compliant.7. Upgrade the VMware Tools version in your virtual machines (if

only upgrading Virtual Hardware, proceed to step 8):a. From the VMs and Templates Inventory page, select the Upgrades

folder, and click the Update Manager tab.b. Click Remediate. The Remediate wizard appears.c. On the Remediation Selection page, select the VMware Tools

Upgrade to Match Host (Predefined) baseline, then click Next.d. Check that all the virtual machines are selected and click Next.e. On the Schedule page, accept the defaults and click Next.f. On the Rollback Options page, clear the Take a snapshot of the

virtual machines before remediation to enable rollback option checkbox, then click Next.

g. On the Ready to Complete page, review the upgrade informationand click Finish.

8. Upgrade VM Hardware to match the host in your virtual machines:a. From the navigation pane of the VMs and Templates Inventory

page, select the Upgrades folder. Click the Update Manager tab.


b. Click Remediate. The Remediate wizard appears.c. On the Remediation Selection page, select VM Hardware Upgrade

to Match Host (Predefined).d. Check that all the virtual machines are selected and click Next.e. On the Schedule page, accept the defaults and click Next.f. On the Rollback Options page, clear the Take a snapshot of the

virtual machines before remediation to enable rollback check box,then click Next.

g. On the Ready to Complete page, review the upgrade informationand click Finish.

9. When remediation finishes, check that all of the virtual machinesare compliant with the attached baselines.

10. Select each virtual machine and click the Summary tab. Check thatthe Virtual Hardware version is the latest version, and that VMwareTools has a value of Current.

Upgrading virtual infrastructure softwareThe upgrade of virtual infrastructure software is a task that should beplanned very carefully. The process involves multiple stages and eachstage needs to be performed in a particular order. When an upgradeis required use the vSphere Upgrade Guide from VMware, Inc. for thetarget release version of vSphere.

When upgrading to the next major release of vSphere always upgradethe infrastructure software in the following order:

1. vCenter Server2. Management ESXi host3. Production ESXi hosts4. VMware Tools5. Experion Backup and Restore

ATTENTION: It is important to ensure that the upgrade isplanned as vCenter Server and ESXi host downtime is expectedduring the upgrade.

Production ESXi hosts

1. Upgrade the production ESXi hosts using vSphere UpdateManager.

202


203

ATTENTION: Do not upgrade the ESXi hosts until firstupgrading the vCenter Server and Update Manager. Theprocess of upgrading an ESXi host requires the ESXi host tobe restarted. Therefore, prior planning is required for thistask.

To upgrade the production ESXi hosts:

a. Download a host upgrade bundle.The bundle can be in the form of a full installation ISO, or asubset of system patches. These packages are available fordownload from www.honeywellprocess.com.

b. Creating a host upgrade baseline.c. Scan and remediate the ESXi host.For more information, see the VMware vSphere Update ManagerInstallation and Administration Guide.

VMware Tools

1. Upgrade the VMware Tools on each virtual machine.This process will require a restart of each virtual machine so thisshould be planned and timed to occur with the Windows updates.For more information, see Installing and Administering VMwarevSphere Update Manager.

Management ESXi host

1. Upgrade the management ESXi host using the manualremediation process.

ATTENTION: Do not upgrade the ESXi hosts until firstupgrading the vCenter Server and Update Manager. Theprocess of upgrading an ESXi host requires the ESXi host tobe restarted. Therefore, prior planning is required for thistask.

This task may require the vCenter Server and Update Manager tobe shutdown also, so the normal method of using UpdateManager to update the management host is not possible.To upgrade the management ESXi host:



a. Download a host upgrade bundleb. Manually remediate the management hostc. Restart the management hostd. Ensure that vCenter Server is running

Experion Backup and Restore

For backup storage guidance, refer to the Experion Backup andRestore (EBR) documentation.

Maintaining hardware devices in a virtualizationenvironment

Maintaining the level 2.5 routerThis topic provides guidance for replacing or updating the virtualinfrastructure level 2.5 router. Internetwork Operating System (IOS) isused in Cisco Systems routers and network switches.

Prerequisites

n Fault Tolerant Ethernet Overview and Implementation Guide.

n Knowledge of the router configuration information, such asHostName, Enable secret, Passwords, IP addresses, and subnetmasks.

n For replacing a failed router:

l Image of the IOS matching the IOS version of the failed router.

l Level 2.5 router configuration file for the failed router.

n For replacing/upgrading existing routers:

l Image of the most current IOS version.

l Level 2.5 router configuration files created from the latestHoneywell provided templates.

n A physical computer to connect directly to the serial console portof the router.

n PuTTY or Tera Term is installed on the physical computer.

204


205

To replace the Level 2.5 router

1. For instructions on how to replace or update a level 2.5 router, seethe “Replacing Switches” topic in the Fault Tolerant EthernetOverview and Implementation Guide.There are some differences in maintaining a Level 2.5 routercompared to maintaining an FTE switch. However, the rules andmethodology apply.Note the following differences:

l The Level 2.5 router is a switch with router capabilities.Whenever you see the term switch, substitute it with Level 2.5router.

l Honeywell does not qualify the IOS versions for the Level 2.5router.

l The IOS version applied depends on the reason for the update.

l Replacing a failed Router the IOS version should match thefailed router.

l Upgrading existing Router the newest IOS version availableshould be used.

ATTENTION: Older versions of a Router IOS aresometimes not available from the manufacturer. It isgood practice to download and archive a copy of yourcurrent Router IOS version now for future recoverypurposes. Honeywell recomends that you always use thelatest IOS version available.

l Honeywell has not evaluated stacking of the Level 2.5 router.Therefore, stacking information should be ignored.

l The Host Name given to Level 2.5 routers should help inidentifying their Hot Standby Router Protocol (HSRP) Primaryand Secondary roles.For example, a host name of level 2.5_Yellow and level 2.5_Green or level 2.5_Primary and level 2.5_Secondary isrecommended.

l Honeywell provides two template config files for the Level 2.5router and they are called level 2.5Yconfig_std.txt and level2.5Gconfig_std.txt. The install package for these files can be


downloaded from the Honeywell Process Solutions web siteand is located with the other FTE Switch configuration files.

l SNMP configuration is not required for the Level 2.5 router butmay be optionally configured to meet your projectrequirements.

Monitoring the virtualization environmentThe vSphere virtual environment provides built-in tools that providethe ability to monitor the resource usage of the virtual infrastructureand signal alarms as to the status of the virtual infrastructure whenthresholds are crossed. It is important to understand these tools sothat the health of the virtual environment is maintained.

It is also possible to use the System Status display in Experion Stationto monitor the various virtual infrastructure status alarms describedin "About system status" on page 211 .

The virtual environment may require monitoringwhen operators using PlantCruise experience slow display call-up orinconsistent refresh rates on displays. Using the Station DisplayPerformance table in the PlantCruise Station Specification documentas a guide, monitor the Station display update rate and display call uptimes. Compare the specification limitations and call up times to thevirtual Station performance and ensure that performance is within thespecification limits.

Engineering tools can also give an indication that the virtualenvironment may require monitoring. When the PlantCruiseengineering tools are used, any inconsistency in the time to performactions indicate that the resource usage of the virtual infrastructuremay not be optimal.

To rectify performance issues seen when comparing update rate andcall-up times with the specification and when engineering tools showinconsistency, see "About resource usage" below and "About systemstatus" on page 211.

About resource usageVirtual infrastructure resource usage is monitored using theperformance charts in the vSphere Web Client. These charts helpadministrators view the resource usage and performance indicatorsin the virtual environment. Guidance on the usage of both theoverview performance charts and the advanced performance

206


207

charts are provided in the “Monitoring Inventory Objects withPerformance Charts” section.

To monitor resource usage for a virtual object, that object must berunning. For example, performance charts for a VM are only availableif the VM is powered on and running.

The five performance areas that define the health of the virtualenvironment are:

n CPU

n Disk I/O

n Memory

n Network

n Storage

CPU performance

CPU usage of virtual machines and the virtual infrastructure is a veryimportant consideration in the overall performance on the system.Monitoring the CPU usage is the best way to ensure that performancedegradation is not occurring. Usage of the advanced CPUperformance charts helps to give the best understanding of thecurrent and past system CPU usage.


Virtualmachine CPUusage

When monitoring CPU usage on a virtual machine, select the virtualmachine and view its advanced performance charts. Select the real-time CPU chart with Usage and Usage in MHz selected. The chartshows the last hour of CPU usage in relation to percentage and MHzused. Ensure that the CPU usage is not constantly above 90%.Occasional spikes up to 100% are acceptable. For more information,see the "Solutions for Consistently High CPU Usage" section invSphere Monitoring and Performance.

VirtualmachineCPUcontention

When monitoring CPU contention on a virtual machine, select thevirtual machine and view its advanced charts. Select the real-timeCPU chart with Ready selected. The chart shows the last hour of CPUready time. Ensure that the ready time does not run constantly above2000ms, and that spikes do not exceed 4000ms. For moreinformation, see the "Solutions for Consistently High CPU Usage"section in vSphere Monitoring and Performance.


Disk I/O performance

Disk I/O performance should be considered whenever monitoringvirtual machines or the virtual infrastructure. Monitoring disk I/Ousage will help give the best indication of the health of the systemsdisk arrays.


Virtualmachine disklatency

When monitoring virtual machine disk latency, select the virtualmachine and view its advanced performance chart. Select the real-time Datastore chart with Read latency and Write latency selected.Ensure that the virtual machine disk I/O latency runs below 25ms.Occasional spikes above 25ms are acceptable. For more information,see the "Solutions for Disk Performance Problems" section invSphere Monitoring and Performance.

Virtualmachine diskusage

When monitoring Virtual machine disk I/O usage, select the virtualmachine and then view its advanced performance chart. Select thereal-time datastore chart with Average write requests per second andAverage read requests per second selected. There is no performancethreshold for this chart type. As a general rule the sum of averageread and writes should be below the maximum IOPs and averageIOPs as documented in the HPS Virtualization Specification.

Memory performance

Memory usage in the PlantCruise virtual environment should not haveperformance issues if the amount of allocated memory is not greaterthan the physical memory in the host. Monitoring the usage of thismemory can be done using the advanced performance charts.


ESXi hostmemorycontention

When monitoring ESXi hosts memory for contention select the ESXihost in question and view its advanced performance chart. Select thereal-time Memory chart with Balloon and Swap Used selected. Ensurethat the ESXi host always has zero balloon and zero swap used usageas shown in the charts.

ESXi hostmemoryusage

When monitoring ESXi hosts memory for usage select the ESXihost in question and view its advanced performance chart. Select thereal-time Memory chart with Active, Consumed and Granted selected.Ensure that the ESXi host active memory is always less than 90%.

208


209


Virtualmachinememoryusage

When monitoring virtual machine memory select the virtualmachine in question and view its advanced performance chart.Select the real-time memory chart with Active, Balloon, Consumed andGranted selected. Ensure that the balloon is always zero. Granted andconsumed memory are normally the same.

Network performance

Monitoring the virtual network usage is important asbandwidth usage on virtual machines and physical networkuplinks are potential causes of performance degradation. Ensuringthat the potential network bottle necks always have available overheadis important. Network usage can be monitored using the advancedperformance charts.


ESXi hostnetworkusage

When monitoring ESXi hosts network usage select the ESXi host inquestion and view its advanced performance chart. Select the real-time Network chart with all physical vmnics used by the productionnetwork selected in the objects selection and Transmit packetsdropped, Receive packets dropped, Data receive rate and Data transmitrate selected in the counters selection. Ensure that all Transmitpackets dropped and receive packets dropped show zero. View the Datareceive rate and the Data transmit rate trends and ensure thatthe average network usage is less than half of the total availablebandwidth for the network connection.

Virtualmachinenetworkusage

When monitoring Virtual machine network usage select the virtualmachine in question and view its advanced performance chart.Select the real-time Network chart with the virtual machinename selected in the objects selection and Data receive rate and Datatransmit rate selected in the counters selection. Ensure that thevirtual machine does not use excessive bandwidth compared to theavailable network bandwidth supplied by the physical switchconnection to the ESXi host.

Storage

Monitoring of storage performance is the same as disk I/Operformance for virtual machines. Use disk I/O to gauge the


performance of the storage for virtual machines. The use of theadvanced performance charts for storage adaptor and storage pathavailable when an ESXi host is selected give a view to the performanceof the disk I/O from a different perspective inside the host. Thesedifferent views into the storage performance allow the performanceto be viewed from different path and adapter levels to help track downissues.


Datastoreperformance

When the total datastore performance or datastore usage statisticsare required select the ESXi host in question and view itsperformance chart. Select the Storage Path Real time trend then therequired runtime name in the objects selection (this can bedetermined by viewing the ESXi host summary and viewing theproperties on the datastore and clicking on the manage pathsbutton) and select the Read latency, Write latency and Averagecommands issued per second in the counters selection. Ensure thatRead latency and Write latency do not run higher than 25ms.Occasional spikes above 25ms are acceptable. Using the Averagecommands issued per second helps you to establish the IOPs on thewhole datastore.

Disk arrayperformance

When the total disk array performance or disk array usage statisticsare required select the ESXi host in question and view itsperformance chart. Select the Storage Adapter real time trendthen the required adapter in the objects selection (this can bedetermined by using the Configuration > Storage Adapters devicecommand) and select the Read latency, Write latency and Averagecommands issued per second in the counters selection. Ensure thatRead latency and Write latency do not run higher than 25ms.Occasional spikes above 25ms are acceptable. Using the Averagecommands issued per second helps you to establish the IOPs on thedisk array.

Storage used When the total used space on each ESXi host datastore is requireduse the ESXi host Configuration > Storage command to see the totalcapacity and free space on each datastore. The Storage Views tabalso give a good indication of the used space and any space used bysnapshots. Always ensure that the total used capacity on eachdatastore is below 75% as the default vSphere warning for datastoreusage is 75% and the datastore will display a persistent alarm if thisthreshold is crossed.

210


211

About system statusThe status of the virtual infrastructure is known through the usage ofvSphere alarms. These alarms notify that specific events haveoccurred and that the virtual infrastructure may be in a state thatrequires attention. For more about these alarms, see the “Monitoringevents, alarms and automated actions” section of vSphere Monitoringand Performance.

Standard Alarms

vCenter is installed with a number of default alarms that warn you ofresource usage status. The list of alarms shown below should be usedas a warning that PlantCruise virtual machine performance is likely tobe affected:

n Virtual machine CPU usage - Shows a warning at 75% and an alertat 90% - warns that the virtual machine is approaching its CPUlimit. No action should be taken unless CPU is constantly reaching100%

n Virtual machine memory usage - Shows a warning at 85% and analert at 95% - warns that the virtual machine is actively using itsallocated memory and the guest operating system could potentialstart to use swap files instead of memory. When virtual machinememory is above 95% for extended periods of time virtualmachine memory should be increased.

n Host CPU usage - Shows warning at 75% and an alert at 90% -warns of ESXi host CPU constraints that may be detrimental toperformance of the PlantCruise virtual machines. Virtual machinesshould be moved off the ESXi host if host CPU usage exceeds90% for extended periods of time.

n Host memory usage - Shows warning at 90% and an alert at 95%- warns of ESXi host memory constraints that could lead toballooning and memory swapping of PlantCruise virtual machines.ESXi host memory should be increased or virtual machines shouldbe moved off the ESXi host if host memory exceeds 90% forextended periods of time.

Custom Alarms

The following alarm can be configured to warn that virtual machineperformance is likely to be affected:


n Virtual machine CPU ready time - Set Alarm Type to monitorVirtual machines - Set trigger type to virtual machine CPU readytime (ms) - Set warning is above 1800 for 5 minutes and set alertis above 2000 for 5 minutes. This will warn of CPU contention on avirtual machine. When Console Stations are used in a large systemwhere the number of Console Stations is approaching 20 thisvalue should be reduced to set warning is above 800 for 5 minutesand set alert is above 1000 for 5 minutes.

n Virtual machine total disk latency - Defaults to show warning at50ms and alert at 75ms - warns of virtual machine diskcontention. Can indicate that another virtual machine is usingabnormally high disk IOPs or that the disk array is running in adegraded state. As soon as a PlantCruise virtual machine indicatesin alarm action should be taken to fix the disk array performanceor move virtual machines off the datastore to reduce the load. IfIOPs limits are set on the Virtual Machine, consider raising thelimit.

ATTENTION: For better warning of disk performance issuesthe warning trigger can be adjusted to 25ms.

Host health

The status of the ESXi host hardware can also be monitored throughvSphere. The hardware Status Tab is presented in the vSphere WebClient when the vCenter Hardware Status plug-in is installed andenabled. This plug-in allows the hardware status to be monitored. Formore information, see the “Monitoring Host Health Status” section ofvSphere Monitoring and Performance.

Monitoring the virtualization hardware using DellOpenManage Server AdministratorDell OpenManage Server Administrator provides a systemsmanagement interface for the Experion Essentials virtualizationhardware platform. It allows system managers to administer thehardware locally and remotely via a network connection.

Refer to the Essentials platform release note for guidance oninstallation of Dell OpenManage Server Administrator.

212


213

Monitoring the virtualization infrastructure fromExperion StationThe various virtual infrastructure status alarms described in “Aboutsystem status” can be monitored from the System Status Display inExperion Station.

To configure the VMware vCenter Server

1. In the VMware vSphere Web Client Home page click vCenter.2. From the Inventory Lists click vCenterServers then click your

vCenter Server.3. Click Manage>Settings>General>Edit>SNMP receivers and configure

two receivers by clicking the Enabled checkbox next to the secondreceiver and enter the following details:Primary Receiver URL: hostname or IP address of Experion Server AReceiver 2 URL: hostname or IP address of Experion Server BReceiver port: 162 (recommended UDP port number)Community string: public

To configure the Alarm Definitions

1. From the Manage page click Alarm Definitions.2. To edit an existing alarm click that alarm then click Edit.3. To add a new alarm click the + symbol.4. Click General and use the following details to define:

Alarm name: Enter an appropriate name for new alarms onlyDescription: Enter an appropriate description for new alarms onlyMonitor: Select the infrastructure type to be monitored by thisalarmMonitor for: Choose to monitor specific conditions or event.Enable this alarm: Enable this alarm within vCenter Server byclicking the checkbox.

5. Click Triggers and delete existing (X) or add (+) new triggers usingthe following details:Trigger if: Choose whether ANY or ALL of the following conditionswill trigger this alarm.Trigger: Click the combobox and choose the required trigger fromthe drop down list


Operator: Click the combobox and choose the required operatorfrom the drop down listWarning Condition: Click the combobox and choose the requiredcondition from the drop down listCritical Condition: Click the combobox and choose the requiredcondition from the drop down list

6. For each alarm that is required to be seen in the System Statusdisplay in Experion Station, click Actions and add (+) one trigger;Honeywell recommends the following details:Action: Click the combobox and choose Send a notification trap.Configuration: leave blankOK>Warning: leave blankWarning> Critical: Click the combobox and choose Once

Critical>Warning: leave blankWarning>OK: leave blankRepeat actions every: Set to 0 minutes

7. Click Finish.

NOTE: Honeywell recommends creating an alarm for each of the followingtriggers.

Alarm Name Description

vSAN Health ServiceAlarm for overall healthsummary

Default alarm to monitor changes for the overall healthsummary

Cannot connect tostorage

Default alarm to monitor host connectivity to storagedevice

Cannot find vSphere HAmaster agent

Default alarm to alert when vCenter Server has beenunable to connect to a vSphere HA master agent for aprolonged period

Health status changedalarm

Default alarm to monitor changes to service andextension health status

Host connection andpower state

Default alarm to monitor host connection and power state

Host connection failure Default alarm to monitor host connection failure

214


215

Alarm Name Description

Host CPU usage Default alarm to monitor host CPU usage

Host error Default alarm to monitor host error and warning events

Host memory usage Default alarm to monitor host memory usage

Insufficient vSphere HAfailover resources

Default alarm to alert when there are insufficient clusterresources for vSphere HA to guarantee failover

Migration error Default alarm to monitor if a virtual machine cannotmigrate, relocate, or is orphaned

Network connectivitylost

Default alarm to monitor network connectivity on a virtualswitch

Network uplinkredundancy lost

Default alarm to monitor loss of network uplinkredundancy on a virtual switch

No compatibile host forsecondary VM

Default alarm to monitor if no compatible hosts areavailable to place Secondary VM

Virtual machine CPUusage

Default alarm to monitor virtual machine CPU usage

Virtual machinememory usage

Default alarm to monitor virtual machine memory usage

vSphere HA failover inprogress

Default alarm to alert when vSphere HA is in the processof failing over virtual machines

vSphere HA host status Default alarm to monitor health of a host as reported byvSphere HA

Inventory alarm health Inventory service alarm to monitor health status

To add a device for the vCenter Server in the NetworkTree

1. Start Configuration Studio and connect to the System2. In the Network tree add a new Switch Device with the following

details:Name: VCENTERSERVER (or as required)IP Address: IP Address of the vCenter ServerDescription: vCenter Server SNMP Provider (or as required)


Read Community: publicLeave all other fields at default values.

3. Click OK

4. Assign an Associated Asset as required.

216


APPENDIX

AIf you have chosen physical shared storage for your virtual infrastructure, considerthe content provided in this section for your planning implementation andconfiguration.

Physical Shared Storage environments often also require vendor-specificconfiguration guidance.

NOTE: This section is only applicable to the Essentials Platform. The PremiumPlatform implements shared storage using vSANs.

In this section:

About shared storage 218

Redundancy for physical SANs 218

Distributing datastores and volumes across storage networks 218

Preparing the storage network 220

Configuring the storage area network 221

IMPLEMENTING PHYSICAL SHAREDSTORAGE NETWORKS

217

Appendix A - Implementing Physical shared storage networks

About shared storageUse of physical SANs is supported at Level 3 for DCS and SCADA architectures.Deploying shared storage in either architecture requires a dedicated storagenetwork.

Use of shared storage at Level 2 is offered with the Premium Platform (see theVirtualization with Premium Platform guide for planning and implementationdetails).

Physical Storage Area Network (pSAN)

A pSAN is a physical storage device that is separate to any ESXi host. This devicetypically contains a large number (greater than 10) of high speed disk drives (10Kor 15K rpm). It is connected to one or more ESXi hosts through a dedicatedstorage network, such as Fibre Channel or iSCSI.

Currently, iSCSI is the only supported storage network, over multiple one gigabitEthernet connections.

Redundancy for physical SANsFor redundancy and backup storage guidance, refer to the Experion Backup andRestore (EBR) documentation.

Distributing datastores and volumes across storagenetworks

Splitting the available storage space on a SAN, but not a vSAN, into multiplevolumes helps to distribute virtual machines across SAN volumes and supportsavailability, should a SAN volume go offline.

When configuring shared storage, the storage available within the device needs tobe allocated into units known as volumes. You will create datastores for thevirtualization environment in these volumes. The maximum size of a datastore is 2terabytes (TB). Typically, datastores have a one to one relationship with volumes,which means that volumes should not be sized greater than 2 TB. Even if the totalstorage available in the SAN device is less than 2 TB you should considersubdividing the available storage into multiple volumes.

For a single SAN, Honeywell recommends the following volumes be created:

n Volume 1 - for A Server. Size this volume larger than the expected server disksize.

n Volume 2 - for B server. Size this volume larger than the expected server disk

218


size.

n Volume 3 - for all other nodes. Size this volume larger than the disk size of allthe virtual machines expected on this volume.

n Volume 4 - for virtual machine creation. Size this volume larger than the disksize of all the virtual machines you expect to create simultaneously.

For multiple physical SANs, Honeywell recommends the following volumes becreated:

n Volume 1 - for nodes that always run on this SAN device, such as PlantCruiseservers, domain controllers, redundant OPC servers, and other redundantnodes.

n Volume 2 - for nodes that can be run on any SAN device, such as Flex Stations,and non-redundant nodes.

n Volume 3 - for backups of virtual machines on volume 1 from another SAN.

n Volume 4 - for an asynchronous replica of volume 2 from another SAN.

n Volume 5 - for virtual machine creation. Size this volume larger than the disksize of all the virtual machines you expect to create simultaneously.

ATTENTION: Depending on the size of the volume, number of virtualmachines, and the number of SAN devices you may choose to furthersubdivide volume 1 and volume 2. For each subdivided volume 1 andvolume 2 there needs to be a matching subdivided 'volume 3' and 'volume4' on another SAN device.

Example 2 SAN design

For a two (2) SAN design, the replication pattern maylook like the following diagram.

Figure A-1: Example 2 SAN design

219


ATTENTION: Volume 5 is not shown as it is notbacked up or replicated to another SAN.

Preparing the storage networkPrepare the storage network, including installing the storage network switch andnetwork cabling, and configuring the storage network switch.

NOTE: For shared storage, four status IP addresses are required for eachESXi host that is connected to the storage network.

Prerequisites

n You have followed the manufacturer's instructions and setup the storagenetwork switch on the required subnet. You have also followed the bestpractices supplied by the manufacturer for connecting and configuring theSAN with VMware.

n vSphere Installation and Setup

To prepare the storage network

1. Configure storm control on the storage network switches.

2. On the storage network switches, enable the following for each interface thatis an end point interface (for example, iSCSI):

220


a. Port fast.

b. BPDU guard.

c. Switch port access.

ATTENTION: In this document, an endpoint interface refers to thetermination of the connection. Therefore, an endpoint is not an uplink toanother switch or network.

3. Enable jumbo packets on both the storage network switches and the networkinterface cards (NICs).

4. Use the default rapid spanning tree protocol (RSTP). On Cisco switches, this isPVRSTP+ or MST.

5. Set the speed and duplex settings wherever possible (avoid auto negotiate).

6. Open the ports identified in vSphere Installation and Setup.

Configuring the storage area networkIf you are using shared storage, configure the virtual storage network on the ESXihost and configure the virtual switch to communicate with the physical storageNICs.

ATTENTION: Only software iSCSI initiators are supported. iSCSI HBAs arenot supported.

Prerequisites

n Storage network NICs are installed on the ESXi host.

n At least 2 static IP addresses for the storage network have been allocated perESXi host.

n Storage network NICs, switches, and network cabling are connected andconfigured.

n ESXi host added to the vCenter Server.

n Refer to SAN Vendor guidance for iSCSI connectivity with vSphere. An exampleis: Dell Equallogics - Configuring iSCSI Connectivity with VMware vSphere 5and Dell EqualLogic PS Series Storage TR1075.

221


Network configuration for hosts with shared storageThe minimum three ESXi host network configuration for a SCADA architectureusing shared storage. Flex Station and Engineering Station thin clients connect tothe production network, while any physical vCenter Client nodes connect to themanagement network. The backup storage device is for the backup of virtualmachines on the ESXi hosts. The storage network connects the ESXi hosts to thestorage area network (SAN).

An ESXi host using shared storage and containing only management workloads,connects to both the management network and the shared storage network.

Figure A-2: Example configuration of a management ESXi host with shared storage

Network configuration for production hosts with sharedstorage (using a storage network)

The minimum three ESXi host network configuration for a SCADA architectureusing shared storage. Flex Station and Engineering Station thin clients connect tothe production network, while any physical vCenter Client nodes connect to themanagement network. The backup storage device is for the backup of virtualmachines on the ESXi hosts. The storage network connects the ESXi hosts to thestorage area network (SAN).

Figure A-3: Example topology of a SCADA architecture using shared storage

222


In the following figure vmnic2 and vmnic3 have been teamed, and connect asingle production vSwitch (vSwitch1) to the physical production network andswitch. Virtual machines connect through the iSCSI Initiator to a vSwitch(vSwitch2) to connect to the storage network.

Figure A-4: Example configuration of an ESXi host using shared storage containingprocess operational workloads

Production Network

Management Network

ESXi Management

Production ESXi Host Details with Shared Storage for SCADA Architecture

vmnic 0

vmnic 1

vmnic 2

vmnic 3

Storage Network Avmnic 5

Storage Network Bvmnic 4

vSwitch 1

vSwitch 2

iSCSI Initiator

Experion server A vNIC 0

vSwitch 0

OPC server vNIC 0

Flex Station vNIC 0

Flex Station vNIC 0

In the following figure, vmnic2 and vmnic3 connect to separate vSwitches(vSwitch1 and vSwitch2 respectively), which connect to the dual Ethernetproduction network. Virtual machines connect through the iSCSI Initiator to avSwitch (vSwitch2) to connect to the storage network.

223


Figure A-5: Example configuration of an ESXi host with local storage containingprocess operational workloads and using dual production networks

Production Network B

Management Network

ESXi Management

Production ESXi Host Details with Shared Storage for SCADA Architecture

vmnic 0

vmnic 1

vmnic 2

Storage Network Avmnic 5

Storage Network Bvmnic 4

vSwitch 1

vSwitch 3

iSCSI Initiator

vSwitch 0

Production Network Avmnic 3

Experion server A

OPC server

Flex Station

Flex Station

vNIC 0vNIC 1

vNIC 0vNIC 1

vNIC 0vNIC 1

vNIC 0vNIC 1

vSwitch 2

224

225

NOTICES

Trademarks

Experion® and SafeBrowse® are registered trademarks of HoneywellInternational, Inc.

PlantCruise™ is a trademark of Honeywell International, Inc.

Other trademarks

Microsoft and SQL Server are either registered trademarks ortrademarks of Microsoft Corporation in the United States and/orother countries.

Trademarks that appear in this document are used only to the benefitof the trademark owner, with no intention of trademark infringement.

Third-party licenses

This product may contain or be derived from materials, includingsoftware, of third parties. The third party materials may be subject tolicenses, notices, restrictions and obligations imposed by the licensor.The licenses, notices, restrictions and obligations, if any, may be foundin the materials accompanying the product, in the documents or filesaccompanying such third party materials, in a file named third_party_licenses on the media containing the product, or athttp://www.honeywell.com/ps/thirdpartylicenses.

Documentation feedback

You can find the most up-to-date documents on the HoneywellProcess Solutions support website at:http://www.honeywellprocess.com/support

If you have comments about Honeywell Process Solutionsdocumentation, send your feedback to: [email protected]

Use this email address to provide feedback, or to report errors andomissions in the documentation. For immediate help with a technicalproblem, contact your local Honeywell Technical Assistance Center(TAC).

Notices

http://www.honeywell.com/ps/thirdpartylicenses

http://www.honeywellprocess.com/support

mailto:[email protected]

How to report a security vulnerability

For the purpose of submission, a security vulnerability is defined as asoftware defect or weakness that can be exploited to reduce theoperational or security capabilities of the software.

Honeywell investigates all reports of security vulnerabilities affectingHoneywell products and services.

To report a potential security vulnerability against any Honeywellproduct, please follow the instructions at:

https://honeywell.com/pages/vulnerabilityreporting.aspx

Support

For support, contact your local Honeywell Process SolutionsCustomer Contact Center (CCC). To find your local CCC visit thewebsite, https://www.honeywellprocess.com/en-US/contact-us/customer-support-contacts/Pages/default.aspx.

Training classes

Honeywell holds technical training classes that are taught by processcontrol systems experts. For more information about these classes,contact your Honeywell representative, or seehttp://www.automationcollege.com.

226

Notices

https://honeywell.com/Pages/vulnerabilityreporting.aspx

https://www.honeywellprocess.com/en-US/contact-us/customer-support-contacts/Pages/default.aspx

https://www.honeywellprocess.com/en-US/contact-us/customer-support-contacts/Pages/default.aspx

http://www.automationcollege.com/