it handbook - storage: the backbone of your virtual environment

Storage: The Backbone ofyour Virtual Environment

q EVALUATING STORAGEFOR VIRTUAL ENVIRONMENTS

q TESTING STORAGEFOR VIRTUALIZATION

q EXPLORING ADVANCEDSTORAGE FEATURES

q TROUBLESHOOTING STORAGE

EVALUATING STORAGE FORVIRTUAL ENVIRONMENTS

TESTING STORAGEFOR VIRTUALIZATION

EXPLORING ADVANCEDSTORAGE FEATURES

TROUBLESHOOTINGSTORAGE

2 STORAGE : THE BACKBONE OF YOUR V IRTUAL ENV IRONMENT

Evaluating Storagefor Virtual Environments

with server virtualization, operating systems and applications are no

longer tied to one specific piece of hardware. They canmove frommachine to

machine—either manually or automatically—and their data must be able to

follow. Shared storage makes that happen.

So what exactly is shared storage? It refers to a device or devices that store

data for and are accessible by multiple systems over a network. In addition to

providing the flexibility needed to support virtualization, shared storage also

increases efficiency.When servers don’t have to spend the resources to store

data locally or transmit data to attached devices, they can use that processing

power to handle more workloads—or more intensive workloads—which helps

server consolidation.

There are twomain storage approaches for virtualization: storage area net-

works (SANs), which typically use iSCSI or Fibre Channel technologies, and

network-attached storage (NAS). Each technology has its pros and cons. Your

organization’s goals, size and budget—among other factors—will determinewhich

technology or blend of technologies is a good fit for your virtualization project.

FIBRE CHANNEL SAN

A SAN uses block-level storage, which relies on a filing system to write and re-

trieve data. It provides high performance, but it can be expensive and complex.

Cost and complexity can be prohibitive for small andmedium-sized busi-

nesses for whom the costs of buying, installing andmanaging a SANmay out-

weigh the savings that would result from consolidating a few physical servers.

Despite these issues, SAN technology is still a popular choice for virtualization

storage, especially among larger organizations. That’s because many SANs rely

on Fibre Channel networking, which provides high-speed data transmission.

EVALUATINGSTORAGE FOR

VIRTUALENVIRONMENTS

The speed of data transmission is essential in virtual environments, espe-

cially when it comes to performance. After all, it doesn’t matter how fast a phys-

ical server and its virtual machines (VMs) can process data if the data is slow in

arriving to those systems in the first place. For some workloads, such as test

and development projects, a certain amount

of lag timemay be acceptable. But for oth-

ers, such as databases orWeb servers, max-

imum performance is a must.

When calculating potential return on in-

vestment (ROI) in a Fibre Channel SAN, or-

ganizations should consider speed because

it will affect backup and recovery. Some

businesses start losing money the second a

certain application goes down, and the

bleeding doesn’t stop until the application comes back up. In these cases, speed

is of the utmost importance, and the ROI of Fibre Channel is clear. But for or-

ganizations that can tolerate some downtime, Fibre Channel might not be

worth the money.

FIBRE CHANNEL OVER ETHERNET

For organizations looking to realize the benefits of Fibre Channel with less cost

and complexity, Fibre Channel over Ethernet (FCoE) is an emerging alternative.

The FCoE protocol lets organizations use their existing Ethernet infrastruc-

tures to transmit data across a SAN. Because of that, it requires fewer cables

and network cards, which can reduce the cost of a SAN implementation.

Despite these benefits, FCoE does have some drawbacks. The speed of exist-

ing Ethernet networks has raised concerns among some administrators. But as

more organizations move from 1-Gigabit Ethernet to 10-Gigabit Ethernet, that

becomes less of an issue.

Other businesses are hesitant to add storage traffic to their existing Ethernet







VIRTUALENVIRONMENTS

The speed of datatransmission isessential in virtualenvironments, espe-cially when it comesto performance.

networks because of the addedmanagement it requires—and the friction it can

cause between storage administrators and network administrators.

iSCSI SAN

Another SAN option is iSCSI, a protocol that arrived after many organizations

had already adopted Fibre Channel. Like FCoE, iSCSI can run over Ethernet

networks, and that can result in cost savings and reducedmanagement com-

plexity.

Organizations that choose iSCSI can use their existing networks without

having to buy the expensive switches, adapters and other equipment required

to support Fibre Channel. Plus, administrators don’t have to learn how to man-

age a whole new network.

Some experts say that performance takes a hit on an iSCSI SAN compared to

Fibre Channel. Others say that a properly configured iSCSI SAN can reach sim-

ilar performance levels.

NETWORK-ATTACHED STORAGE

The SAN versus NAS debate is similar to the Fibre Channel versus iSCSI de-

bate in that each centers on both cost and performance.

Unlike a SAN, NAS takes a file-level approach to storage for virtualization.

The additional layer of abstraction that a SAN provides isn’t there with NAS, so

there’s less complexity and, therefore, lower costs.

But security is an added concern when using NAS for virtualization storage.

Because NAS stores the file systems, anyone who can access the NAS device

can access those files if they have the proper privileges. SANs control security

themselves, regardless of the security levels of individual file systems.

Having options can help organizations come up with the best solutions. In

the next segment, learn how to test each approach and determine which is best

for your environment. �







VIRTUALENVIRONMENTS

Testing Storage for Virtualization

you can do all the research in the world, but the only way to make sure you

have the right storage for your virtual environment is to test it. It’s important to

test your storage to find out which results require the most attention.

As of now, the use of solid-state drives (SSD) is not commonplace. Even if it

were, all the available shared storage technologies communicate over a wire

from the virtualization host to a storage array. This wire either communicates

using Fibre Channel protocol, iSCSI, a network file system or Fibre Channel

over Ethernet.

These protocols and the speeds at which the data is transmitted across the

wire are often seen as limiting factors in storage. Also consider all the devices

on the path to the disk where the data will eventually be saved. This includes

such things as host bus adapters, local cache, Ethernet or Fibre Channel

switches, storage processors, disk controllers and the spindles on each of the

disks, if you are not using SSD.

STORAGE AS A SHARED RESOURCE

In addition to the hardware considerations, remember that storage in a virtual

environment is a shared resource—shared not only among VMs running on a

host but also among hosts. So any technology used for storage—whether file

sharing or block storage—must support clustered file systems. That implies

some ability to minimally lock the file system table of contents to ensure up-

dates are serialized across all hosts.

This type of locking often adds quite a bit of complexity into a seemingly sim-

ple storage system. Now add the ability to have multiple storage paths from the

VM to the storage device, and you will see that real-world testing is often re-

quired in the form of a bake-off between storage vendors.

The tests are intended to determine two things: one is performance of reads






TESTINGSTORAGE FOR

VIRTUALIZATION

and writes through the virtual environment—often referred to as latency—and

the other is the maximum number of I/O operations per second (IOPS) that can

take place to and from the storage device without increasing latency. IOPS can

also be seen as a measure of the number of VMs a particular storage device can

handle per logical unit, or LUN, of the storage device.

In some cases, using more LUNswill decrease lock operations. It will also in-

crease IOPSwhen a LUN is locked because no operations can take place for the

length of the lock, which is usually less than 10 microseconds. But because

quite a fewwrites or reads can take

place during 10 microseconds, it’s im-

portant to remember that IOPS and la-

tency values will suffer if the lock

extends past the desired time.

CREATING A REAL-WORLD

TEST ENVIRONMENT

When setting up a test for your storage

subsystem, create as close to a real-

world test as possible. Ideally, you

would put the new storage system into

your QA lab and thenmove via Stor-

age vMotion or otherwise migrate

your real test environment to the new storage device.

If your QA environment matches your production environment, you have a

ready-built test environment. If your QA environment does not match your

production environment, then you have your work cut out for you.

For better results, youmay consider trying to match your QA environment as

closely as possible to your production environment in your count of application

instances, quantity of transactions and data per transaction. If your QA envi-

ronment does not match your production environment, your storage latency






TESTINGSTORAGE FOR

VIRTUALIZATION

If your QA environmentmatches your productionenvironment, you havea ready-built test envi-ronment. If your QAenvironment does notmatch your productionenvironment, then youhave your work cut outfor you.

and IOPS values as well as counts of VMs per LUNwill not be accurate and

may actually be quite a bit worse than expected when you finally place the cho-

sen storage device into production.

TWO TESTS TO CONSIDER

There are at least two tests to consider: those that have many small virtual disks

per LUN and those that have a few large virtual disks per LUN. If you are test-

ing Tier 1 applications, youmay also have raw disk maps for very large LUNs to

hold data such as a database or an extremely large file share.

Ensure that your test mimics the real

world as much as possible. Determine

your max throughput by inputting trans-

actions in your environment as fast as

you possibly can. Next, use a real-world

quantity of transactions to determine

your steady-state numbers.

It is important to use the proper tools

to determine latency and IOPS values.

Such tools will be able to get performance

data directly from the array as well as de-

termine how your virtual environment

sees the performance—both are required. Once you have completed your test-

ing, you can then choose the proper layout of your storage within the virtual en-

vironment and the proper storage subsystem. The key for this type of testing is

good planning, suitable documentation and a good test suite that is as real-

world as possible. �






TESTINGSTORAGE FOR

VIRTUALIZATION

Once you havecompleted your testing,you can then choosethe proper layout ofyour storage withinthe virtual environ-ment and the properstorage subsystem.

Exploring Advanced Storage Features

storage can enhance your virtual environment. Adding advanced features

such as data deduplication, thin provisioning and others can actually improve

virtualization performance and reduce your storage footprint.

Storage is one of the more expensive parts of any virtual environment. Be-

cause of that, use this limited resource sparingly. The good news is that today’s

technology helps administrators use storage more efficiently.

Some technologies that aid efficiency are in hardware, such as data dedupli-

cation. Others are not, such as thin provi-

sioning and linked clones. All these

technologies are designed to allow ad-

mins to use limited storage capabilities

better.

HOW DATA DEDUPLICATION WORKS

Data deduplication allows a storage de-

vice to condense identical blocks of data

down to one block with a link back to

that block per LUN and sometimes

across LUNs. This happens within the storage processor and is possible only

because of advancements in processor capabilities such as multiple cores.

In some cases, data deduplication happens after the fact as a background

task. Until the deduplication task is completed, the number of actual blocks of

disk being usedmay change.

Deduplication is extremely useful for unencrypted data because once the

data is encrypted, deduplication would be minimal.While already removing

like blocks from a file, compressed data may still participate in deduplication

because many compressed files of similar construct can still be deduplicated.






EXPLORINGADVANCEDSTORAGE

FEATURES

Data deduplicationallows a storage deviceto condense identicalblocks of data down toone block with a linkback to that block perLUN and sometimesacross LUNs.

Encryption attempts to avoid having the data appear the same twice, regardless

of similarities.

On the other hand, thin provisioning is a technology that allocates only the

in-use blocks of a file or virtual disk instead of the completely requested alloca-

tion of blocks. Thin provisioning could work with data deduplication to use

even fewer blocks of disk than previous thought because instead of allocating

blocks to participate in deduplication, the blocks are not allocated until needed.

Unfortunately, there is no way to

prevent a thin provisioned file or vir-

tual disk from allocating all its blocks.

Because of that, either could easily use

muchmore storage than originally en-

visioned.

ADVANTAGES OF LINKED CLONES

Linked clones provide the ability to

link one virtual disk to another virtual disk in such a way that only the changes

between the original virtual disk and the cloned virtual disk are recorded

within the clone. This is an extremely useful construct if a base part of a virtual

disk does not change, such as in the operating system of a VM.

The linked clone would instead contain only those changes required for a

given application, user or service. Linked clones tied to thin provisioning and

data deduplication could lead to quite a few storage blocks saved on a storage

device.

But a linked clone can grow to the full size of the original disk if the blocks of

the original disk are changed in any way, such as the case of an operating sys-

tem patch or disk optimization performed fromwithin the linked clone.

Linked clones have a further limitation in the depth of clones. The more

depth to the clones, the slower the virtual disk will be because of the block fix-

ups that are required.







FEATURES

Linked clones tied tothin provisioning anddata deduplicationcould lead to quite a fewstorage blocks saved ona storage device.

WEIGHING STRENGTHS AND WEAKNESSES

If used correctly, all of these technologies could reduce the overall quantity

of storage required for your virtual environment, but each has its ownweak-

nesses and strengths. One weakness is that to use vSphere Fault Tolerance, the

virtual disk must not be thin provisioned or a link cloned.What is required is a

thickly allocated disk, meaning that all blocks are allocated and in use.

Data deduplication will not produce the same results if data is encrypted

before being written to the storage

array. That is the only way it would hap-

pen with today’s technology.

Always use care with tools such as

thin provisioning and linked clones be-

cause these technologies can easily be

abused. Any savings you thought you

would receive would be incorrect be-

cause of sudden growths of the virtual

disks to full sizes.

The truly inconsiderate component of

these sudden growths is that technology

does not yet exist to tell you that this is eminent. The virtual environments re-

port the full size of the virtual disks instead of the allocated sizes, and the stor-

age devices report only on what space is left, not on what space is available on a

per-file basis.

So use extreme care and audit results when using these technologies. Some

form of continual monitoring may be required if space on your storage arrays

suddenly grows.

Thin provisioning and linked clones are ways to fit more virtual disks per

LUN and to make better use of your storage device. Alternately, data deduplica-

tion has no such limits unless the data is encrypted. The bottom line is to use

caution when choosing these technologies, and know their limitations. �

1 0 STORAGE : THE BACKBONE OF YOUR V IRTUAL ENV IRONMENT






FEATURES

Data deduplicationwill not produce thesame results if data isencrypted before beingwritten to the storagearray. That is the onlyway it would happenwith today’s technology.

Troubleshooting Storage

problems are bound to pop up nomatter howwell you have researched and

implemented storage in your virtual environment. Uptime and I/O speed can

be challenging to maintain. The best defense is to find out how to locate the

source of possible problems and fix them.

In general, once you have a properly configured storage subsystem that is

built to meet the requirements of your virtual environment, there is often no

need to modify the environment unless something changes. Those changes can

include adding more applications to the original mix or adding new application

functionalities that require more storage.

Some changes could be more subtle. Connection issues happen occasionally,

and usually the first to fail is the virtualization host. Modern virtualization

hosts are good at connecting to storage that has been properly presented and

zoned to the virtual environment.

If there is a connectivity issue, it is essential to first check the hardware in-

volved for proper connections—such as making sure lights are properly lit—

and then check the zoning and presentation of the LUN to the virtualization

host. It should be noted that some hypervisors limit the number of LUNs that

can be used.

LOCKING STORAGE SUBSYSTEMS

Locking is required when dealing with files that are shared and that cross host

boundaries. So, at a minimum, the table of contents of each file system should

be locked.

When virtualization first started, locking storage subsystems was a prevalent

problem. But with modern hypervisors, many of these issues have been solved

by applying evenmore technology such as segmented table of contents, in-

creased lock time-outs and increased number of lock retries.







Storage subsystems have also improved, but problems still arise occasionally.

Amore prevalent issue is SCSI reservation time-out, which means the lock

failed so the data was only partly written or not written at all.

No one wants to see a time-out error message in a vSphere log file because it

implies something went seriously wrong. Before reviewing all disk write ac-

tions that have taken place, don’t forget to inspect the hardware.

Because most people assume the hardware is not at fault, a hardware inspec-

tion is often performed last. But, for locking issues, the reverse is often true.

After inspecting the hardware, review other types of actions that may cause

locks. Investigate anything that changes the table of contents of the clustered

file system.

Performance issues are those stemming from the measured or perceived

slowness of an application. The first thing to understand about performance is

that it may not be the storage subsystem, but you’ll need the proper tools to de-

termine if this is actually true.

WHICH TOOLS TO USE

Modern virtual environments have several built-in management tools that will

show the LUN latency values as well as the number of IOPS in use. Many third-

party tools also give the same data. Getting this data as close to the hardware is

often paramount to determining what is actually going on.

One helpful tool is NetApp Balance, which correlates VM activity with the

activity on the LUN of the questionable storage array. If latency is high, then re-

balance the data across LUNs and perhaps across storage paths until there is a

better latency score.

Changing storage paths could also imply moving the load from one virtual-

ization host to another. For these tasks, vSphere Storage vMotion and vMotion

or Xen/Hyper-V LiveMigration tools can be extremely useful.

There is no easy fix for a large number of IOPS unless you have control

over the application and can spread the load across multiple LUNswith data-







splitting or caching technologies or can use massively parallel databases such

as Greenplum. But even with technologies like these, the solution is often to re-

vamp an application instead of bolting on a solution.

And vSphere’s Storage IO Control (SIOC) functionality could also be used to

give higher priority to writes made by those applications marked as critical or

those that have an increased amount of shares of the storage path than other

virtual disks on the same path.

But SIOC is a contention-based tech-

nology based on LUN latency values. If

the latency values are below 30mi-

croseconds, then SIOC is not in use.

Multipath-plugin tools are also help-

ful. They allow you to use tools provided

by the storage vendors to aggregate, load

balance and better control how data is

written to the storage array.

There is also an increase in using lo-

calized SSD drives and storage cache

cards to improve overall write and read times to and from the storage array.

Writes hit the local cache and then are forwarded over the wire. Local reads of

the same data are then accelerated, as is the response time for writes.

Other issues can cause a malfunction in storage environments, such as a sud-

den loss of connectivity or storage processor failures. These failures often show

up as virtual environment issues andmay be investigated within the virtual en-

vironment first.

Any failure will adversely affect your virtual environment. Storage failures

can show up as other issues that suddenly increase latency and affect perform-

ance It’s always a good idea to investigate the storage subsystem at the same

time. �







Multipath-plugintools allow you to usetools provided by thestorage vendors toaggregate, load balanceand better control howdata is written to thestorage array.

Edward L. Haletky is the authorofVMware vSphere and VirtualInfrastructure Security: Securingthe Virtual Environment andVM-ware ESX and ESXi in the Enter-prise: Planning Deployment ofVirtualization Servers, 2nd Edi-tion. He owns AstroArch ConsultingInc., which provides virtualization,security, network consulting and de-velopment, and The VirtualizationPractice, where he is an analyst.Haletky is the moderator and host ofthe Virtualization Security Podcastand moderator for the VMware Com-munities Forums.

Colin Steele is senior site editorfor SearchServerVirtualization.comand SearchVMware.com, which pub-lish expert content for IT profession-als working with server virtualiza-tion technology. Before joiningTechTarget in 2007, he was a news-paper reporter for The Eagle-Tribune in North Andover, Mass.Steele has previously written for theGloucester Daily Times, Tri-TownTranscript and The Daily Orangeat Syracuse University, where hereceived his bachelor’s degree innewspaper journalism in 2002.






EXECUTIVE EDITORJo Maitland

SITE EDITORColin Steele

SENIOR MANAGING EDITORSMichelle BoisvertLauren Horwitz

MANAGING EDITORChristine Casatelli

ASSOCIATE MANAGING EDITORSJeannette BeltranEugene Demaitre

Martha Moore

DIRECTOR OF ONLINE DESIGN

Linda Koury

EDITORIAL DIRECTORCathleen Gagne

PUBLISHERMarc Laplante

[email protected]

TechTarget275 Grove Street, Newton, MA 02466

www.techtarget.com

©2011 TECHTARGET. ALL RIGHTS RESERVED.

ABOUT THEAUTHORS

mailto:[email protected]

RESOURCES FROM OUR SPONSOR

• How vStorage Enabled SANs Will Help Maximize VMware Investment

• Virtualizing Microsoft Applications with VMware vSphere on Dell Servers and Dell EqualLogic Storage

• Calculating Storage Requirements: Impact of Solid State Drives, Virtualization, and CloudComputing

http://content.dell.com/us/en/enterprise/by-partner-vmware

http://www.bitpipe.com/resource/1303225407_166





it handbook - storage: the backbone of your virtual environment

Technology

virtualization storage

storage administrators

storage traffic

noevaluating storage

storage nas

fibre channel san

transmission istesting

storage area net