exanodes-vmware esx-quick start guide

8/2/2019 Exanodes-VMware ESX-Quick Start Guide

1/18

Exanodes Virtual Machine Edition 1.1

for VMware ESX

Quick Start Guide


2/18

Exanodes Virtual Machine Edition 1.1 for VMware ESX: Quick Start

Guide

e06-VM-1.1 rev01

Publication date 04/20/2009

Copyright 2009 Seanodes

Abstract

This Guide contains information on how to quickly setup the Exanodes application. It is a step-by-

step, task-oriented guide for rapid deployment of the Exanodes solution.

Disclaimer and Copyright

The information contained in this publication is subject to change without notice. SEANODES makes no warranty of any kind

with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular

purpose. SEANODES shall not be liable for errors contained herein or for incidental or consequential damages in connection

with the furnishing, performance, or use of this manual.

All Rights Reserved. SEANODES, Exanodes, are registered trademarks of SEANODES. Other product names mentioned herein

may be trademarks or registered trademarks of their respective companies.


3/18

iii

Table of Contents

1. Introduction ......................................................................................... 1

2. Presentation ......................................................................................... 2

3. Exanodes VM requirements ....................................................................... 4

3.1. Memory ...................................................................................... 4

3.2. Storage ...................................................................................... 4

3.3. CPU ........................................................................................... 4

3.4. Network ..................................................................................... 4

3.5. Time ......................................................................................... 4

4. Installation .......................................................................................... 5

4.1. Configuring the Exanodes VM network ................................................. 5

4.2. OVF package installation procedure .................................................... 7

4.3. ESX package installation procedure ..................................................... 7

4.4. Adding disks to the Exanodes disk pool ................................................ 74.5. Verifying the Exanodes VM setup ....................................................... 8

5. Exanodes VM storage configuration ............................................................ 10

6. VMware ESX configuration ....................................................................... 12

6.1. Setting up the VMware ESX software iSCSI initiator ................................ 12

6.2. Creating a VMFS datastore on the Exanodes storage ............................... 13

7. Guest VM configuration .......................................................................... 14

7.1. Configuring the virtual machines restart ............................................. 14

7.2. Stopping the virtual machines ......................................................... 14

Glossary ................................................................................................ 15


4/18

1

1. Introduction

This document illustrates the necessary steps to install the Exanodes VM.

This document does not explain how to manage Exanodes; please refer to the Exanodes

User's Guide for more information about Exanodes management. It does not either cover the

detailed instructions for setting up a virtual machine.

This document provides indications on the integration of the Exanodes solution into a Virtual

Infrastructure environment. For more information about VMware configuration, please check

the documentation available with your VMware software package.

Throughout this document, we will use VMware vCenter Server (previously: Virtual Center

(VC)) and Virtual Infrastructure Client (VIC) 2.5.0 to perform administrative tasks. Please

refer to the appropriate VMware documentation on how to setup VMware vCenter Server.

Security Note: Seanodes states that the Exanodes VM does not contain any virus, spyware

or malware known to this date.

Exanodes virtual machine edition is designed to be compatible with almost every type of

commoditized x86 server supported by VMware ESX 3.5 or ESXi.

Table 1.1. Compatibility list

Server

CPU i386, x86_64, AMD, Intel

500MHz minimum

Memory 256MB minimum

Network 2 Gigabit Ethernet network cards

Internal disks Full disk, disk partition

Internal RAID, DAS (or SAN storage with exclusive server access)

SSD

Hypervisor

Supported ESX

versions

ESX 3.5

ESXi

Other: Please contact SEANODES


5/18

2

2. Presentation

The following presents the way Exanodes should be implemented in a VMware virtualized

environment. The following illustration is the "big picture" of what happens on one ESX

hosting an Exanodes VM.

Figure 2.1. A typical Exanodes VM Storage stack on one ESX host

Local physical disks (1) are used for 2 purposes: storing the local Exanodes VM on the ESX (one

and only one Exanodes VM per ESX host) and offering the storage pools that the Exanodes

VM will use

Each Exanodes VM must be capable of accessing other Exanodes VMs through the network to

work properly and to bring the level of functionalities and fault tolerance Exanodes offers

to the storage.

VMFS uses physical disks (2) also called VMHBA devices where data is actually stored, and

upon which (3) datastores are created. Virtual disks are then created on these datastores

for Exanodes use.

Exanodes (4) aggregates these virtual disks (3) from many hosts (from the VMHBA physical

devices (2)). It then virtualizes them to create volumes that are exported through an iSCSI

target (5).

The storage is then imported back in ESX through the iSCSI software initiator (6), to

participate to the VMFS storage pool as LUNs. Once formatted as a VMFS datastore, any VM

virtual disk (7) can be created on this storage. They can be used by the application VMs (8)


6/18

Presentation

3

The following figure illustrates this mechanism at the cluster level:

Figure 2.2. A typical Exanodes VM cluster

Each ESX host runs one and only one Exanodes VM. Thanks to the Shared Internal Storage,

each ESX has access to the same same datastores exported by the Exanodes iSCSI target. So

any VM (like VM3) can be started on any ESX host and migrated to any ESX host.

Note

It is mandatory that the Exanodes storage used by other VMs goes through VMFS to benefit

from the VMFS functionalities like VMotion and snapshots. It is not recommended to configure

a VM with direct access to the Exanodes storage via an iSCSI connection.


7/18

4

3. Exanodes VM requirements

3.1 Memory

Each Exanodes VM must have a minimum of 256 MB reserved memory.

3.2 Storage

Each Exanodes VM should be configured with one or more independant - persistent virtual

disks for Exanodes VM use. It is not mandatory to assign disks to each Exanodes VM, but

the best practice is to provide a homogeneous disk setup between the Exanodes VMs. i.e.

assign virtual disks of the same size and if possible located on the same physical disk type

to each Exanodes VM.

3.3 CPU

Each Exanodes VM needs a minimum of 500 MHz reserved CPU or more for performance.

3.4 Network

The network requirements for Exanodes VM iSCSI are the same as with a traditional iSCSI

storage system. Each Exanodes VM must be able to get its IP information over DHCP, to

keep the same IP, hostname and DNS information after reboot. Adapt the network settings

accordingly. A good way of doing this is to set a constant MAC address for each Exanodes VM

and have the DHCP configuration set-up for each of these MAC addresses.

At the hardware level, Exanodes requires using a non-blocking switch that supports multicast.

Note: The Exanodes VM must use a dedicated Virtual Machine Network bound to a dedicated

physical network adapter. Sharing this Virtual Machine Network with other VMs or VMKernel

is NOT SUPPORTED.

3.5 Time

It is recommended to have ESX time set-up correctly, as the VMware tools in the Exanodes

VM make use of it to synchronize time. Setting-up ESX using NTP is a good practice.


8/18

5

4. Installation

It is considered that all disks that will either store the Exanodes VM or the storage pool, are

already configured by the ESX host as part of a VMFS datastore. The Exanodes VM installation

procedure depends on whether you are using the OVF version or the ESX version.

4.1 Configuring the Exanodes VM network

The Exanodes VM requires two dedicated virtual switches:

The first vSwitch is used for the local iSCSI loopback, it must not be linked to a physical NIC

The second vSwitch is used for the Exanodes network traffic, it must be connected to a

physical NIC to allow access to the other Exanodes VMs on the other hosts.

First configure the dedicated iSCSI loopback virtual switch. This means creating a VM Network

(for the Exanodes iSCSI target), and a VMkernel & Service console to enable the ESX iSCSI

software initiator to contact the Exanodes target:

1. Click on the ESX host Configuration tab

2. Hardware Networking

3. Click on "Add networking" link at the top right of the window

4. Select "Virtual Machine"

5. Select "Create a virtual switch" and uncheck the physical NIC if necessary

6. Type "VM Network for Exanodes iSCSI" as the new virtual network label

7. Next Finish

8. Click on the "Properties" of the new vSwitch created (at the top right of this vSwitch)

9. Click "Add"

10. Select "VMkernel"

11. Change the label to "VMkernel for Exanodes iSCSI"

12. Set the IP and submasks addresses to "10.0.0.1" and "255.255.255.0"

13. You may have to set the default gateway if this was not previously done

14. Click "Next", then "Finish"

15. Click "Add"

16. Select "Service Console"

17. Change the label to "Service Console iSCSI"

18. Set the IP and submasks addresses to "10.0.0.2" and "255.255.255.0"

19. Click "Next", then "Finish"

Then configure the Virtual Network dedicated to Exanodes VM:

1. Click on the ESX host Configuration tab

2. Hardware Networking

3. Click on the "Add networking" link at the top right of the window


9/18

Installation

6

4. Select "Virtual Machine"

5. Select "Create a virtual switch" and choose the NIC dedicated to Exanodes VM

6. Type "VM Network for Exanodes" as the new virtual network label

7. Next Finish

Reminder: this new "VM Network for Exanodes" port group MUST NOT be used by other VM

appliances nor its vSwitch be configured with a VMKernel port.

For better performance, SEANODES recommends that other appliances use a dedicated

virtual machine network bound on a third NIC.

The next screenshot shows a typical Network Configuration, with only 2 physical NICs.

Figure 4.1. A typical ESX network configuration


10/18

Installation

7

4.2 OVF package installation procedure

Unzip the Exanodes VM package using your favorite UnZip tool (the document you are reading

is part of this zip file).

1. File Virtual Appliance Import

2. Choose "Import from File" and select the OVF file : ExanodesVM.ovf

3. Next Next

4. Read and Accept the License Agreement Next

5. Provide a name "ExanodesVM-" for example

6. Choose a local datastore to store the Exanodes VM (Exanodes Root Disk)

7. Choose "VM Network for Exanodes" in the first "Network label" combo box

8. Choose "VM Network for Exanodes iSCSI" in the second "Network label" combo box9. Next Finish

4.3 ESX package installation procedure

Upload the directory called ExanodesVM-VERSION which contains .vmx and .vmdk files

to your ESX server in the appropriate VMFS datastore. It contains the minimal requirements

for the Exanodes VM to work. Then from the virtual infrastructure client:

1. Select the ESX configuration tab and then the storage item

2. Right-click on the local datastore where you wish to put this directory and select Browse

Datastore

3. Click on the "upload" icon. Choose "Upload Folder" and select the ExanodesVM-

VERSION folder of the Exanodes ESX distribution

4. Browse the newly uploaded DISK folder. Right click on ExanodesVM.vmx and select

'Add to inventory'

5. In the VMware installation steps: Provide a name "ExanodesVM-" for

example

6. If no VM Network called "VM Network for Exanodes" exists, specify the one to use

Check that the Virtual Network Adapter uses the Virtual Network dedicated to Exanodes.

4.4 Adding disks to the Exanodes disk pool

Now, you must add virtual disks to the pool of storage you wish Exanodes to virtualize:

1. Right Click on ExanodesVM

2. Edit Settings

3. Add

4. Click on "Hard Disk"


11/18

Installation

8

5. Choose "Create a new Virtual Disk"

6. Choose "Specify a Datastore" and choose the datastore

7. Set the appropriate size, and click on Next

8. Within the Advanced Options, set the disk mode to: Independent - Persistent

9. Next, verify the new device summary and click on Finish to add the disk.

You must repeat these last operations for any virtual disks you wish to virtualize using

Exanodes.

Note

It is not recommended to virtualize virtual disks stored on VMFS volumes made of multiple

extents stored on different disks.

The following screenshot shows an Exanodes VM vmware41v1.vmdomain, running on the

ESX host sam41.toulouse.com.

Figure 4.2. A typical Exanodes VM configuration

4.5 Verifying the Exanodes VM setup

1. Right Click on ExanodesVM Edit Settings (Properties)


12/18

Installation

9

2. Hardware Tab Memory 256 MB

3. Set all virtual disks (the ExanodesVM system and data disks) to independent/persistent.

4. Resources Tab CPU

5. Right panel: Shares "High" , Reservation 500 MHz

6. Memory: Reservation 256MB

7. Disks: Shares "High" for all virtual disks

8. Hardware Tab Network Adapter 1: Network label "VM Network for Exanodes" (or

the dedicated one you created)

9. Network Adapter 2: Network label "VM Network for Exanodes iSCSI" (or the dedicated

one you created)

You can now start the Exanodes VM: Right Click on ExanodesVM Power On

Once the VM is running, you can log on using the "root" or "exanodes" accounts, the defaultpasswords being "exanodes".

You must repeat all these operations (the Exanodes VM installation, disks, the Exanodes VM

setup) on all the ESX hosts on which you wish to have access to the Exanodes storage.


13/18

10

5. Exanodes VM storage configuration

At this point, you should be able to log into any of your Exanodes VM hosts using either the

console or SSH.

SSH access is possible with the "exanodes" user, the default password being "exanodes". Once

logged on you can run the Exanodes CLI commands to configure the cluster and storage.

Please refer to the Exanodes User's Guide for more detailed information on configuring

clusters, groups and volumes.

The following command examples achieve this:

1. First create the cluster. node/1-3/ represents the 3 nodes of the cluster, whose

hostname are: node1 node2 node3/.

[exanodes@node1 ~]$exa_clcreate mycluster --node node/1-3/ --really-create

Creating the cluster mycluster:

** ----------------------------------------------------------------------- **

** This command erases all the disks listed in your configuration. **

** Please check carefully your disks are free to be used by Exanodes **

** ----------------------------------------------------------------------- **

Hit Ctrl-C if unsure. Remaining seconds : 0

Checking nodes are not already used SUCCESS

Sending cluster configuration SUCCESS

2. Then start it:

[exanodes@node1 ~]$exa_clstart mycluster

Initializing the cluster (Please wait):

Initializing the cluster SUCCESS

3. Create a disk group with all available disks and start it:

[exanodes@node1 ~]$exa_dgcreate mycluster:mygroup --layout rainX --all -s

Creating disk group 'mycluster:mygroup':

Disk group create: SUCCESS

Starting disk group 'mycluster:mygroup'

Disk group start: SUCCESS

4. Once you've finished with Exanodes groups, you can configure Exanodes volumes. Each

Exanodes volume is mapped to an iSCSI LUN device by an iSCSI target (which is part

of Exanodes). It is therefore important to plan in advance the number of volumes and

their sizes.

[exanodes@node1 ~]$exa_vlcreate mycluster:mygroup:vol1 --size 100G

Creating a 100 G volume 'mygroup:vol1' for cluster 'mycluster'

Creating volume 'mygroup:vol1': SUCCESS

5. Starting a volume on a node will present the volume to the ESX host running that node.

Start each volume on the nodes where access from the ESX hosts to the volume is

required. Starting a volume exports it as an iSCSI LUN. Here, we start the volume on

all nodes.


14/18

Exanodes VM storage configuration

11

[exanodes@node1 ~]$exa_vlstart mycluster:mygroup:vol1 --all

Starting volume 'vol1' in the group 'mygroup' for cluster 'mycluster'

Volume start: SUCCESS

Next, we need to configure access to the Exanodes iSCSI target for each ESX host that requires

it.


15/18

12

6. VMware ESX configuration

6.1 Setting up the VMware ESX software iSCSI initiator

Perform the following steps to setup the software iSCSI initiator on the ESX host:

1. Click on the ESX host and select the configuration tab

2. Select the Storage Adapters item

3. From iSCSI software adapters select 'Properties' on the lower panel

4. On the 'General' Tab select configure, check 'Enable' and click 'OK'

5. On the 'Dynamic Discovery' Tab select 'Add'

6. Enter the IP address of the iSCSI target. This is "10.0.0.3" by default in the Exanodes VMunless you manually changed the network configuration. Click 'OK'

7. The initiator is probing the target click on 'Close'

8. Right Click on the 'iSCSI software Adapter' icon and select the 'Rescan' action

9. The initiator detects the LUN started inside the local Exanodes VM

Figure 6.1. The list of iSCSI targets

Note

Although all ESX hosts are configured with the same target IP address, each will contact their

local Exanodes iSCSI target through the local loopback.

The following screenshot shows that 3 LUNs of 10GB and 5GB and 3GB have been detected

after rescan:


16/18

VMware ESX configuration

13

Figure 6.2. The list of detected LUNs

You must perform these operations on all hosts that need to access the Exanodes storage.

6.2 Creating a VMFS datastore on the Exanodes storage

You can now use the Exanodes storage as any other storage, which means you can format it

and import it in VMFS. To do this, follow these steps, not specific to Exanodes:

1. Click on the ESX host and select the configuration tab

2. Select 'Storage'

3. Click on 'Add storage'

4. Choose 'Disk / LUN'

5. Choose the LUN you wish to use on this ESX click 'Next' twice

6. Provide a name for the datastore

7. Click Next twice and then Finish


17/18

14

7. Guest VM configuration

The storage we configured earlier is ready to use, but due to the inner way in which it is

accessed (from Exanodes, which is a VM), there are a few parameters that must be set.

7.1 Configuring the virtual machines restart

As the iSCSI storage is presented by Exanodes which runs inside a VM, only this VM will be

able to boot at start-up. There is no auto-detection of new LUNs on an iSCSI connection, so

it's necessary to run a rescan of the target shown by the Exanodes VM after a reboot, and

only after the Exanodes VM has completed its initialization. After this, you can restart the

Virtual Machines residing on Exanodes storage.

7.2 Stopping the virtual machinesWe must configure any VM using the Exanodes storage to stop before the Exanodes VM.

Click on the ESX and select the Configuration Tab

In "Software", Virtual Machine Startup / Shutdown

Check "Allow Virtual Machines to start and stop automatically with the system"

Set the Exanodes VM to start first, stop last, and a delay of a few minutes


18/18

15

Glossary

Cluster A group of interconnected servers (nodes).

Node A system belonging to a cluster.

Disk A disk is a storage system able to read or write data to

a storage device: hard drive, partition, volume, ram disk,

network drive... It can be accessed as a block device from the

operating system.

Disk group A disk group is a storage subsystem that creates a storage array.

A disk group agglomerates the storage space from all of its

disks.

Layout The layout defines the way Exanodes will arrange (locate) the

data on a disk group.

RainX RainX is a disk group data layout where redundant information

is split between the disks, allowing for the loss of one disk

while providing full access to data. Moreover, it has optional

support for spare disks. Each spare disk allows the RainX layout

to handle an additional disk loss, provided there is sufficient

time in between to allow Exanodes to rebuild the data. The

RainX layout requires at least 3 nodes sharing one disk (or more)each.

Simple striping Simple Striping is a disk group data layout, where data is striped

across the disks, resulting in higher data throughput. Since no

redundant information is stored, failure of a disk in the group

can result in data loss.

Volume A volume is a virtual storage area. The storage space of a

disk group is used to create volumes up to the total physical

capacity of the group.

exanodes-vmware esx-quick start guide

Documents