emulex - management mind meld (a. ordoubadian)

Ke

y Te

rm

ino

log

y

SAn

/lAn

Tec

hn

olo

gy

ove

rvi

ew

h

igh

AvA

ilAb

iliT

y/FA

ulT

To

ler

Ance

P

er

For

mAn

ce

Sec

ur

iTy

m

AnAg

em

en

T

ManageMent Mind Meld What net Work adMinistrators need to knoW about storage ManageMent

What NetworkAdministrators Need

to Know AboutStorage Management

What Network Administrators Need to Know about Storage Management

iWhat Network Administrators Need to Know about Storage Management i

AbstractData center administrators face a major networking challenge from the combination of high bandwidth requirements, increasing network sprawl and the need for a more adaptive networking infrastructure. Most data centers today have:

• Multiplenetworkfabrics,eachdedicatedtoaspecifictypeoftraffic

• Highnumbersofadaptersandswitchportdeployments

• Complexcablinginfrastructure

• Complexmanagementofswitchandadapterfirmwareandassociatedservice contracts

Data centers are implementing a new consolidated network technology for data andstorage,called“convergednetworking.”ConvergednetworkingcombinesexistingLocalAreaNetworks(LANs)andStorageAreaNetworks(SANs)intoasingle,high-performance10Gb/sEthernet(10GbE)frameworkthatintelligentlyconnects every server, network and storage device within the data center, therebyenablingunifiedI/O.

Converged networking results in an overlap of network and storageadministrators’ responsibilities. This guide explains networking and storagebasics to help each administrator better understand the changes resulting from converged networking and how it will impact their role in the data center. The following sections are provided in this guide:

• IntroductionwithKeyTerminology:GeneralSAN/LANTechnologyOverview

• HighAvailability/FaultTolerance

• Performance

• Security

• Management

• EmulexComponents

• Conclusion

What Network Administrators Need to Know about Storage Managementiiii

iiiWhat Network Administrators Need to Know about Storage Management iii

Table of ContentsAbstract i

Chapter 1: Evolution of the Data Center 1DriversforNetworkConvergence. . . . . . . . . . . . . . . . . . . . . . 1TheDataCenterNetworkingChallenge. . . . . . . . . . . . . . . . . . . 1

Chapter 2: 10 Gigabit Ethernet, the Enabling Technology for Convergence 3

Chapter 3: Technology Overview 5FibreChanneloverEthernet . . . . . . . . . . . . . . . . . . . . . . . . 5FibreChannelCharacteristicsPreserved . . . . . . . . . . . . . . . . . . 6iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Chapter 4: Storage Area Networks 8SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8LogicalUnitNumber . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9FibreChannelProtocol . . . . . . . . . . . . . . . . . . . . . . . . . . 10LayersofFibreChannelProtocol . . . . . . . . . . . . . . . . . . . . . 10InternetFCP(iFCP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11OSIModelvs.FC/FCoE . . . . . . . . . . . . . . . . . . . . . . . . . 11WorldWideName . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Converged Networking 12DataCenterBridging(DCB) . . . . . . . . . . . . . . . . . . . . . . . 12PriorityFlowControl(PFC) . . . . . . . . . . . . . . . . . . . . . . . 12EnhancedTransmissionSelection(ETS) . . . . . . . . . . . . . . . . 13

HowFCoETiesFCProtocolwithNetworkProtocol. . . . . . . . . . . . 13RequirementstoDeployLoss-lessEthernet . . . . . . . . . . . . . . . 13NonFibreChannelBasedStorageProtocols . . . . . . . . . . . . . . . 13

Chapter 5: SAN Availability 14KeyTerminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14SANTrunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15FailoverandLoadBalancing . . . . . . . . . . . . . . . . . . . . . . 15

ConfiguringFailoverinaSAN . . . . . . . . . . . . . . . . . . . . . . . 16EffectofConvergedNetwork . . . . . . . . . . . . . . . . . . . . . . . 16QoS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17DataCenterBridgingeXchange(DCBX) . . . . . . . . . . . . . . . . 17Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

What Network Administrators Need to Know about Storage Managementiv

Chapter 6: Performance 18SANperformanceandcapacitymanagement . . . . . . . . . . . . . . 18EffectofConvergedNetwork . . . . . . . . . . . . . . . . . . . . . . . 18IndustryBenchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 19StoragePerformanceCouncil(SPC) . . . . . . . . . . . . . . . . . . 19TransactionProcessingPerformanceCouncil(TPC) . . . . . . . . . . 19

BenchmarkingSoftware . . . . . . . . . . . . . . . . . . . . . . . . . 21Iometer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21IOzone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21IxiaIxChariot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

KeyTerminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22CPUEfficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

PerformanceTuning . . . . . . . . . . . . . . . . . . . . . . . . . . . 23DriverParameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Queue depth setting . . . . . . . . . . . . . . . . . . . . . . . . . . 23Interruptcoalescing . . . . . . . . . . . . . . . . . . . . . . . . . . 23

KeyMetrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24IOPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Chapter 7: Security 26SecurityinConvergedNetworkingEnvironments . . . . . . . . . . . . . 26SecurityBreaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27MethodsofProtectingaSAN . . . . . . . . . . . . . . . . . . . . . . . 28

Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28VirtualSAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29LUNMasking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30SecurityProtocols . . . . . . . . . . . . . . . . . . . . . . . . . . . 30EncapsulatingSecurityPayloadoverFibreChannel . . . . . . . . . . 31SecuringiSCSI,iFCPandFCIPoverIPNetworks . . . . . . . . . . . 31

EffectofConvergedNetwork . . . . . . . . . . . . . . . . . . . . . . . 32NativeFCoEStorage . . . . . . . . . . . . . . . . . . . . . . . . . . 32Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32LUNMasking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

vWhat Network Administrators Need to Know about Storage Management

Chapter 8: Management: Configuration and Diagnostics 34SANprovisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34AdapterManagement . . . . . . . . . . . . . . . . . . . . . . . . . . 36Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

KeyTerminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37HBAandCNAconfiguration . . . . . . . . . . . . . . . . . . . . . . 37PortConfiguration . . . . . . . . . . . . . . . . . . . . . . . . . . . 38BootfromSAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38vPorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38SMI-S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38CIM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

EffectofConvergedNetwork . . . . . . . . . . . . . . . . . . . . . . . 39FibreChannelInitializationProtocol(FIP) . . . . . . . . . . . . . . . . 39PortConfiguration . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 9: Emulex Solutions 41

Chapter 10: Conclusion 45

What Network Administrators Need to Know about Storage Managementvi What Network Administrators Need to Know about Storage Managementvi

1What Network Administrators Need to Know about Storage Management

Chapter 1: Evolution of the Data Center

Drivers for Network ConvergenceThe combination of high bandwidth demand, increasing network sprawl and the need for more adaptive networking infrastructure is posing a major challenge for datacentermanagers.Painpointsintoday’sdatacenternetworksinclude:

• Multiplenetworkfabrics,eachdedicatedtoaspecifictypeoftraffic (seeFigure1)

• Highnumbersofadaptersandswitchportdeployments

• Complexcablinginfrastructure

• Storagenetworkprovisioningtimesasaresultofstaticconfigurations

• Complexityofmanagingswitchandadapterfirmwareandassociatedservice contracts

HCA

CoreEthernetNetwork

FC SAN

Ethernet Switch

FC Switch

Server

NIC

Infiniband Switch

IB Network

HBA

Figure 1:DedicatednetworksforSANandLAN

The Data Center Networking ChallengeData center managers are clearly in need of networking solutions that contain the sprawl of network infrastructure and enable an adaptive next-generationnetwork.Thesolutionforoptimizingthedatacenternetworkmustbecapableof addressing the following high-level requirements:

1.Consolidate:Thenetworksolutionmustbecapableofconsolidatingmultiple low-bandwidth links into a faster high-bandwidth infrastructure andsignificantlyreducingthenumberofswitchandadapterportsandcables.

2.Converge:Thenetworksolutionmustbecapableofconvergingor

What Network Administrators Need to Know about Storage Management2

unifyingnetworkingandstoragetraffictoasinglenetwork,eliminatingtheneedfordedicatednetworksforeachtraffictype.Thisfunctionalitywillfurther contribute toward reduction in network ports and cables, while simplifying deployment and management.

3.Virtualize:Thenetworksolutionmustbecapableofvirtualizingtheunderlying physical network infrastructure and providing service level guaranteesforeachtypeoftraffic.Inaddition,thesolutionmustbe capable of responding to dynamic changes in network services depending on the business demands of the data center applications.


Chapter 2: 10 Gigabit Ethernet, the Enabling Technology for Convergence

The10GbEnetworkingstandard,ratifiedin2002,enablesmultipletraffictypesoverasinglelink,asshowninFigure2.InordertofacilitatenetworkconvergenceandcarryFibreChanneltrafficover10GbE,Ethernettechnologyhadtosupporta “no-drop” behavior because SAN traffic requires a loss-less transmission.To alleviate the “lossy” nature of traditional Ethernet environments, 10Gb Data CenterBridging(DCB)wasdevelopedtoprovidealoss-lessconnection,makingit ideal for storage networking applications.

10GbEcanoperateboth as a “loss-less” and “lossy” network.Ports canbeconfiguredtocarryvariousprotocols:

• TCP/IP

• InternetSmallComputerSystemInterface(iSCSI)

• FibreChanneloverEthernet(FCoE)

Figure 2:10GbEenablesmultipletraffictypesoverasinglelink

The DCB Task Group of IEEE 802.1 Working Group (LANs) provides thenecessary framework for enabling 10GbE converged networking within a data center. The recent innovations of this task group that support the loss-less characteristicin10GbEaresummarizedbelow:


10GbE Innovations

• Enhancedphysicalmedia

o 10Gb/sconnectivityoverUTPcabling

o 10Gb/sconnectivityoverDirectAttachTwin-axCoppercabling

• Optimizationsin10Gb/stransceivertechnology(SFP+formfactor)

• Supportforloss-lessEthernetinfrastructure

• Newphysicalnetworkdesignssuchastop-of-rackswitcharchitectures

• IsolateandprioritizedifferenttraffictypesusingPriorityFlowControl(PFC)

• Maintainbandwidthguaranteesformultipletraffictypes

• Assurethatend-pointsandswitchesknowabouteachother’scapabilitiesthroughanenhancedmanagementprotocolusingDCB

These innovations rely on the following four key protocols:

Protocols Key Functionality Business Value

PriorityFlowControl(PFC)P802.1Qbb

Management of bursty, singletrafficsourceona multi-protocol link

Enablesstoragetrafficover 10GbE link with “no-drop” in the network

Enhanced Transmission Selection(ETS)P802.1Qaz

Bandwidthmanagement between traffictypesformulti-protocol links

Enables bandwidth assignments per traffictype.Bandwidthisconfigurableon-demand.

DataCenterBridgingCapabilitiesExchangeProtocol(DCBCXP)802.1Qaz

AutoexchangeofEthernet parameters betweenpeers(switchtoNIC,switchtoswitch)

Facilitatesinteroperability by exchangingcapabilitiessupported across the nodes.

CongestionManagement(CM)P802.1Qau

Addressesproblemofsustained congestion, driving corrective action to the edge

Facilitateslargerend-to-end deployment of network convergence.

Table 1:Protocolstandardsareenablingconvergence

Inadditiontoprovidingloweredcosts,10GbEenablesmuch-neededscalabilitybyprovidingadditionalnetworkbandwidth.10GbEalsosimplifiesmanagementbyreducingthenumberofportsandfacilitatingflexiblebandwidthassignmentsforindividualtraffictypes.


Chapter 3: Technology Overview

Fibre Channel over EthernetIn parallel with the emergence of loss-less 10GbE, the emergence of newerstandards,suchastheFCoEstandard,isacceleratingtheadoptionofEthernetasthemediumofnetworkconvergence.FCoEisastandarddevelopedbyINCITST11thatfullyleveragestheenhancedfeaturesof10GbEforI/Oconsolidationinthe data center.

10GbE networks address the requirements of consolidation, convergence andvirtualization.FCoEexpandsFibreChannelintotheEthernetenvironment,combining two leading technologies, FibreChannel andEthernet, to providemore options to end users for SAN connectivity and networking. Networkconvergence, enabled by FCoE, helps address the network infrastructuresprawl, while fully complementing server consolidation efforts and improving theefficiencyoftheenterprisedatacenter.

FCoE is a new protocol that encapsulates Fibre Channel frames within anEthernetframetravelingona10GbEDCBnetwork.FCoEleverages10GbDCBconnections.AlthoughFCoEtrafficsharesthephysicalEthernetlinkwithothertypesofdata traffic,FCoEdatadelivery isensured,as it isgivena loss-lesspriority status,matching the loss-lessbehavior guaranteed in FibreChannel.FCoEisoneofthetechnologiesthatmakesI/Oconvergencepossible,enablingasinglenetworktosupportstorageandtraditionalnetworktraffic.

Figure 3:Abilityoftechnologytomeetneedsofnetworksegments


Fibre Channel Characteristics PreservedThe FCoE protocol specification maps a complete Fibre Channel frame(includingchecksum,framingbits)directlyontotheEthernetpayloadandavoidsthe overhead of any intermediate protocols.

Figure 4:FCoEencapsulationinEthernet)

This light-weightencapsulationensuresthatFCoE-capableEthernetswitchesare less compute-intensive, thus providing the high performance and low latencies of a typical Fibre Channel network. By retaining Fibre Channel astheupperlayerprotocol,thetechnologyfullyleveragesexistingFibreChannelconstructssuchasfabriclogin,zoningandlogicalunitnumber(LUN)masking,and ensures secure access to the networked storage.

Data center managers are looking for solutions to transition to a more dynamically provisioned network that is highly responsive and addresses the quality and service level requirements of business applications.

iSCSIThe iSCSIprotocol ratifiedby InternetEngineeringTaskForce (IETF) in2003broughtSANswithinthereachofsmallandmid-sizedbusinesses.TheprotocolencapsulatesnativeSCSIcommandsusingTCP/IPandtransmitsthepacketsover the Ethernet network infrastructure. The emergence of 10GbE addressed the ITmanager’s concerns regarding the bandwidth and latency issues of 1Gb Ethernet and laid the foundation for more widespread adoption of network convergence in data centers.

iSCSI-enabledconvergenceoffersseveraladvantages:

• Highlysuitableforconvergenceinsmallandmediumbusinesses,remoteofficesanddepartment-leveldatacenterswherecustomersaretransitioningfromDirectAttachStorage(DAS)toSANs.

• Reduceslaborandmanagementcostswhileincreasingreach.

• TheubiquitousnatureofEthernetmeansthatIPnetworkscanbedeployedquicklyandeasilyinorganizationsofallsizes.Ethernetis


alsoreadilyunderstood,soITpersonnelcandeployandmaintainanIPenvironmentwithoutspecializedFibreChanneltraining.

• MajoroperatingsystemsincludeaniSCSIdriverintheirdistribution.

• iSCSIperformancecanbeimprovedbydeployingadaptersthatsupportiSCSIoffloadorTCP/IPoffloadtoreducetheCPUdemandsforpacketprocessing.

Althoughoptimalforsmallandmediumbusinesses,iSCSI-enabledconvergencedoes have limitations:

• BecausetheunderlyingEthernetnetworkispronetopacketlosseswithnetwork congestion, network designers typically recommend the use of separate Ethernet networks for storage and data networking. This reduces some of the cost advantages of convergence.

• LargeenterprisedatacentershaveasizabledeploymentofFibreChannelSANsanduseFibreChannel-specifictoolstoeffectivelymanagestorageassets.Fromtheperspectiveofthesecustomers,iSCSIisadifferent storage technology that requires an incremental investment in hardware, software and training.

ThedecisiontodeployiSCSIorFCoEislargelybasedoncurrentdeployments.Enterprise data centers with Fibre Channel SANs already in place typicallychooseFCoE,whilesmallerdatacenterswithnoFibreChanneltypicallychooseiSCSI.


Chapter 4: Storage Area Networks

Understanding SAN technology requires familiarity with the terms andcomponents described in this section.

SANASANisanarchitecturethatattachesremotecomputerstoragedevices(suchasdiskarrays,tapelibrariesandopticaljukeboxes)toserversinamannerwherethedevicesappear as locally attached to theoperating system (OS).ASANgenerally is its own network of storage devices that are typically not accessible throughtheLANbytypicaldevices.

Historically,byvirtueoftheirdesign,datacentersfirstcreated“islands”ofSCSIdiskarraysasDAS,eachdedicatedtoanapplication,andvisibleasanumberof“virtualharddrives”(i.e.,LUNs,definedbelow).Essentially,aSANconsolidatessuchstorageislandstogetherusingahigh-speednetwork(seeFigure5).

Figure 5:StorageAreaNetwork


CommonusesofaSANincludeprovisionoftransactionallyaccesseddatathatrequire high-speed, block-level access to the hard drives such as e-mail servers, databases and high-usage file servers. Storage sharing typically simplifiesstorageadministrationandaddsflexibility,sincecablesandstoragedevicesdonot have to be physically moved to shift storage from one server to another.

Otherbenefits includetheability toallowserverstoboot fromtheSAN itself.ThisallowsforaquickandeasyreplacementoffaultyserverssincetheSANcanbereconfiguredsothatareplacementservercanusethebootLUNofthefaultyserver. This process can take as little as half an hour and is a relatively new idea beingpioneeredinnewerdatacenters.SANsalsotendtoenablemoreeffectiveandrobustdisasterrecoverycapabilities.ASANcanalsospanadistantlocationenabling more effective data replication implemented by disk array controllers, byserversoftwareorbyspecializedSANdevices.SinceIPbasedWideAreaNetworks(WANs)areoftentheleastcostlymethodoflong-distancetransport,theFibreChanneloverIP(FCIP)andiSCSIprotocolshavebeendevelopedtoallowphysicalextensionofaSANoverovercomingthedistancelimitationsofthephysicalSCSIlayer,ensuringbusinesscontinuanceinadisaster.

The economic consolidation of disk arrays has accelerated the advancement of several features, including I/O caching, snapshotting and volume cloning(BusinessContinuanceVolumes,orBCVs).

Logical Unit NumberIncomputerstorage,aLUNistheidentifierofaSCSIlogicalunit,andbyextension,ofaFibreChanneloriSCSIlogicalunit.AlogicalunitisaSCSIprotocolentitythatperformsclassic storageoperationssuchas readandwrite.EachSCSItargetprovidesoneormorelogicalunits.AlogicalunittypicallycorrespondstoastoragevolumeandisrepresentedwithinanOSasadevice.

In current SCSI, a LUN is a 64-bit identifier. Note that even though named“LogicalUnitNumber,” it isnotanumber. It isdivided into four16-bitpiecesthat reflect a multilevel addressing scheme, and it is unusual to see any but the firstoftheseused.

To provide a practical example, a typical disk array has multiple physicalSCSIports,eachwithoneSCSItargetaddressassigned.Thenthediskarrayis formatted as a redundant array of independent disks, or also known as redundantarrayofinexpensivedisks(RAID),andthenthisRAIDispartitionedinto several separated storage volumes. To represent each volume, a SCSItarget is configured to provide a logical unit. Each SCSI targetmay providemultiple logical units and thus represent multiple volumes, but this does not mean that those volumes are concatenated. The computer that accesses a volumeonthediskarrayidentifieswhichvolumetoreadorwritewiththeLUNof the associated logical unit.

Anotherexample isasinglediskdrivewithonephysicalSCSIport. Itusuallyprovides just a single target, which, in turn, usually provides just a single logical


unitwhoseLUN is zero.This logicalunit represents theentirestorageof thedisk drive.

Fibre Channel ProtocolFibreChannel isahigh-speednetwork technologyprimarilyused forstoragenetworking.ItusesFibreChannelProtocol(FCP)transportprotocoltotransportSCSI commands over Fibre Channel networks. The following provides asummaryofthedifferencesbetweenFibreChannelandEthernet:

• FibreChannelpassesblockdata,similartoFCoEandtalkingtotargetdevices,whereasEthernetpassesfiles/packets.Blockdataismuchlarger and moved in a loss-less manner. Ethernet is smaller and “lossy” and can be sent out of order.

• FibreChanneltalkstotargetdevices(storagedevice),whereasEthernettypicallytalkstootherhosts(servers).Inthestorageworld,thedistinction between the “target” and the “initiator” is important. Ethernet looks as these as one and the same.

• Withstorageconnectivity,thereisafinitenumberofendpoints,whereasinLANs,thereareaninfinitenumberofendpointsthatneedtotalk to each other.

• InaLAN,thebandwidthrequirementtoanyparticularendpointisgenerally much smaller than the bandwidth requirement for storage networks.ThesignificanceofthisfactisthatinaSAN,youhavebetterpredictabilityoftrafficpatternsandrequirements,andyouwouldlikelycreatetrafficzonesbetweenthefinitenumberofhostconnectionsandstorage connections.

Layers of Fibre Channel ProtocolFibreChannelprotocolconsistsoffivelayers.GiventhatFibreChannelisalsoatypeof“networking”protocol,therearesomesimilaritiestotheOpenSystemsInterconnect(OSI)modelusedinnetworks.TheFibreChannellayersarenotedbelow:

Layer Description

FC0This is the physical layer, which covers cables, transceivers, connectors, pin-outs, etc.

FC1 This is the data link layer, which encodes and decodes signals.

FC2This is thenetwork layer,consistingof thecoreofFibreChannel,anddefiningthemainprotocols.

FC3This is the common services layer, which is a thin layer that could, in the future,supportfunctionslikeencryptionorRAID.

FC4This is the protocol mapping layer. This layer encapsulates other protocols suchasSCSIintoaninformationunitfordeliverytothenetwork(FC2)layer.


Internet FCP (iFCP)The iFCPprotocol enables the implementation of FibreChannel functionalityover an IP network, within which the Fibre Channel switching and routinginfrastructureisreplacedbyIPcomponentsandtechnology.Congestioncontrol,errordetectionandrecoveryareprovidedthroughtheuseofTCP(TransmissionControlProtocol).TheprimaryobjectiveofiFCPistoallowexistingFibreChanneldevicestobenetworkedandinterconnectedoveranIP-basednetworkatwirespeeds.

OSI Model vs. FC/FCoE TheOSILayeredModelisanarchitecturalabstractionthathelpstodescribetheoperationofprotocols.Unfortunately,theFibreChannelprotocollayerscannotbemappedtoOSIlayersinastraightforwardmanner.FCoE,whichleveragesthe Fibre Channel protocol, has an inherent awkwardness when applied toEthernet networks, whereas the iSCSI protocol originated from a traditionalEthernet and IP environment. Figure 6 shows themapping of FibreChannellayerstoOSIlayers.

World Wide NameAWorldWideName(WWN)isa64-bitaddressusedinFibreChannelnetworkstouniquelyidentifyeachelementinaFibreChannelnetwork.TheuseofWWNsforsecuritypurposesisinherentlyinsecure,becausetheWWNofadeviceisauser-configurableparameter.

Figure 6:StorageprotocolsmappedtotheOSImodel


Converged Networking

Data Center Bridging (DCB) TheDCBTaskGroupisapartoftheIEEE802.1WorkingGroup.DCBisbasedonacollectionofopen-standardEthernetextensions.ItisdesignedtoimproveandexpandEthernetnetworkingandmanagementcapabilitieswithinthedatacenter.DCBhelps toensuredatadeliveryover loss-less fabrics, consolidateI/OoveraunifiedfabricandimprovebandwidththroughmultipathingatLayer2(theDatalinkLayer).

With DCB, Ethernet will provide solutions for consolidating I/O and carryingmultipleprotocols,suchasIPandFCoEonthesamenetworkfabric,asopposedtoseparatenetworks.Theabilitytoconsolidatetrafficisnowavailablewiththedeploymentof10GbEnetworksduetothefollowingcomponentsofDCB:

1.Priority-basedFlowControl(PFC)–Enablesmanagementofbursty,singletrafficsourceonamultiprotocollink

2.EnhancedTransmissionSelection(ETS)–Enablesmanagementofbandwidthbytrafficcategoryformulti-protocollinks

3.DateCenterBridgingExchange(DCBX)protocol–Allowsauto-exchangeof Ethernet parameters between switches and endpoints

4.Congestionnotification–Resolvessustainedcongestionbymovingcorrective action to the network edge

5.Layer2Multipathing–UsesallbisectionalbandwidthofLayer2topologies

6.Loss-lessService–Helpsensureguaranteeddeliveryserviceforapplications that require it

WithDCB,a10GbEconnectioncansupportmultipletraffictypessimultaneously,while preserving the respective traffic treatments. The same 10GbE link canalsosupportFibreChannelstoragetrafficbyofferinga“nodatadrop”capabilityviaFCoE.

Priority Flow Control (PFC)

PFC is an enhancement to the existing pause mechanism in Ethernet. ThecurrentEthernetpauseoptionstopsall trafficona link;essentially, it isa linkpause for theentire link.UnliketraditionalEthernet,DCBenablesa linktobepartitionedintomultiplelogicallinkswiththeabilitytoassigneachlinkaspecificpriority setting (loss-less or lossy). The deviceswithin the network can thendetectwhether traffic is “lossy” or “loss-less”. If the traffic is lossy, then it istreatedintypicalEthernetfashion.Ifitisloss-less,thenPFCisusedtoguaranteethat none of the data is lost.

Inshort,PFCallowsanyofvirtuallinkstobepausedandrestartedindependently,enabling the network to create a no-drop class of service for an individual virtual


link. ItalsoallowsdifferentiatedQualityofService (QoS)policies for theeightuniquevirtuallinks.PFCisalsoreferredtoasPerPriorityPause(PPP).

Enhanced Transmission Selection (ETS)

ETS is a new standard that enables amore structuredmethod of assigningbandwidth based on traffic class. Thisway, an IT administrator can allocatea specific percentage of bandwidth to SAN, LAN and inter-processorcommunication(IPC)traffic.

How FCoE Ties FC Protocol with Network ProtocolFCoE transports Fibre Channel frames over an Ethernet network whilepreserving existing Fibre Channel management modes. A loss-less networkfabricisarequirementforproperoperation.FCoEleveragesDCBextensionstoaddresscongestion,trafficspikesandsupportmultipledataflowsononecabletoachieveunifiedI/O.

Requirements to Deploy Loss-less EthernetLoss-lessEthernetenvironmentrequiresthemeanstopausethelink,suchasPFC (asdescribed above) in aDCBenvironment. It also requires themeansto tie the pause commands from the ingress to the egress port across the internalswitchfabric.ThepauseoptioninEthernetandPFCinDCBtakecareofprovidingloss-lessEthernetoneachlink.Finally,aloss-lessintra-switchfabricarchitecture is required.

Non Fibre Channel Based Storage ProtocolsiSCSIisanIP-basedstoragenetworkingstandardforlinkingdatastoragearraystoservers. iSCSI, likeFibreChannel, isamethodoftransportinghigh-volumedatastoragetrafficandisdesignedtobeadirectblock-levelprotocolthatreadsand writes directly to storage. However, unlike Fibre Channel, iSCSI carriesSCSIcommandsoverEthernetnetworks insteadofaFibreChannelnetwork.BecauseoftheubiquityofIPnetworks,iSCSIcanbeusedtotransmitdataoverLANs,WANs or the Internet. iSCSI passes block data, similar to FCoE, andcommunicates to target devices.


Chapter 5: SAN Availability

In the event of an unexpected disruption, each IT infrastructure must bedesignedtoensurethecontinuityofbusinessoperations.Fromadatacenterstorageperspective,thismeansthatyourSANfabricmustbeextremelyreliable,for data must be accessible at all times, whether to do scheduled backups or unexpectedrecoveries.

Within the SAN fabric, high availability is needed across adapters, switches,servers and storage. If a problem occurs with any of these components, acombination of aggregation and failover techniques are used to meet availability and reliability requirements. Figure 7 shows an example ofmultipathing andfailoverinaSAN.

Figure 7: Multipathing and failover

Key TerminologyThe following terminology is important to ensuring SAN high availability/faulttolerance:


SAN Trunking

Trunking(alsoreferredtoasaggregation,linkaggregationorportaggregation)combinesportstoformfasterlogicalcommunicationlinksbetweendevices.Forexample,byaggregatinguptofourinter-switchlinks(ISLs)intoasinglelogical8Gb/strunkgroup,youoptimizeavailableswitchresources,therebydecreasingcongestion.

Trunking increasesdataavailability even if an individual link failureoccurs. Insuch an instance, the I/O traffic continues, though at a reduced bandwidth,aslongasatleastonelinkinthetrunkgroupremainsavailable.Althoughthistypeofaggregationrequiresmorecablingandswitchports,itoffersthebenefitof fasterperformance, loadbalancingand redundancy. It isoftenpossible toaggregate links between a host server and switch, or between a storage system andaswitch,orevenbetweenISLs.

Failover and Load Balancing

Failover and loadbalancing in storagenetworksgohand-in-hand.Byhavingmultiple physical connections, a failure in one adapter port or cable won’t completely disrupt data traffic. Instead, data flowcan continue at a reducedspeed until the failure is repaired.

Another benefit ofmultiple physical connections is load balancing.Normally,unrelated physical links can transfer data at independent and frequently unpredictable speeds, allowing a bottleneck on one or more of the physical connections,which, in turn, can impact theoverall performanceof theSAN.Oncemultiple physical connections are aggregated into a logical data path,data can be distributed equally across the member links to balance the load and reduce bottlenecks within the network.

SANfailoverisaconfigurationwheremultipleconnectionsaremade;however,not all of the connections carry data simultaneously. For example, a storagearraymaybeconnectedusingtwo8Gb/sFibreChannellinks,butonlyoneofthe linksmightbeactive.Thesecond link isconnected,but is inactive. If thefirstlinkfails,thedatacommunicationthenfailsovertothesecondlink,allowingcommunication to continue at the same speed until the original connection is repaired.

SAN QoS

The server, adapter, switch and storage array are critical components when attemptingtodeployQoSwithintheSAN.TheoptimumQoSsolutionshouldbebasedonanoverallviewoftheSAN,befullyinteroperableandfocusoncriticalbottlenecks.

Fibre Channel adapters usually have excess bandwidth and short responsetimes, and, as a result, do not impact overall QoS. This is particularly thecase when following best practices and installing multiple adapters for high availability.


StoragearraysareoftenthelimitingfactorforI/Oandareacriticalcomponentforoverallperformancetuning.ArrayQoSisusuallybasedonLUNs.Forexample,high-priorityapplicationscouldbecombinedonaLUNwithRAIDstriping,high-performancedrives,alargeamountofcachememoryandahighQoSpriority.Another LUN could be used to support less critical background tasks withinexpensive,lowerperformancedisksandalowerQoSpriority.Whenusedincombinationwithallofthesevariables,array-basedQoSmanagementcanbeavery effective tool for storage administrators.

Switch-based QoS can be used to prioritize traffic within the SAN. Someswitches provide a variety of options to implement QoS. They include FibreChannel zones, virtual SANs (VSANs) and individual ports. Fibre Channelswitches are designed to be fully interoperable with industry-standard server-to-SAN connectivity adapters. For example, Cisco QoS provides extensivecapabilitiestocreateclassesoftrafficandassigntherelativeweightforqueues.Otherswitcheshavemoreproprietarydesigns.I/Otrafficbetweentheserverandswitch is not likely to be a bottleneck, which is the case with high-performance adapters that usually have surplus bandwidth.

Configuring Failover in a SANAt many levels, IP and storage networks share similar failover configurationsteps.The followingarea fewof thebasicmethods toconfigure failover inastorage environment:

• Serversconfiguredwithadual-portadapterconnectedtoaswitch

- Each port connected to two different switches

- Createvirtualports(vPorts)ontopofthephysicalportsandhavethemassociated with a switch

• Serversconfiguredwithtwodual-portadaptersconnectedtotwodifferent switches

- Each port of an adapter is connected to a port on one of the two switches

- CreatevPortsontopofthephysicalportsandhavethemassociatedwith different switches

• ServerClusters

- Agroupofindependentserversworkingtogetherasasinglesystemtoprovide high availability of services. When a failure occurs on a server within the cluster, resources are rerouted, redistributing the workload toanotherserverwithinthecluster.Serverclustersaredesignedtoincrease availability of critical applications.

Effect of Converged NetworkConverged networking will introduce new technologies and methodologies


that will change data center reliability and business resilience processes. The following describes some of the changes to be considered.

QoS

Networks require much more than just “speeds and feeds.” 10GbE offersincreasedspeedandbandwidth,butyoustillneedtocontrolit.QoStechnologiesare the means by which it can be controlled, and vendors will be providing these technologies for converged networks.

Data Center Bridging eXchange (DCBX)

DCBX is used by DCB devices to exchange configuration information withdirectlyconnectedpeers.Theprotocolmayalsobeusedformisconfigurationdetectionandforconfigurationofthepeer.

Ethernet is designed to be a “best-effort” network. This means data packets may be dropped or delivered out of order if the network or devices are busy. DCBXisanEthernetdiscoveryandconfigurationprotocolthatguaranteeslinkendpointsareconfiguredinamannerthataverts“softerrors.”DCBXenables:

• End-pointconsistency

• Identificationofconfigurationirregularities

• Basicconfigurationcapabilitiestocorrectend-pointmisconfigurations

DCBXprotocol isused for transmissionofconfigurationsbetweenneighborswithinanEthernetnetworktoensurereliableconfigurationacrossthenetwork.ItusesLinkLayerDiscoveryProtocol(LLDP)toexchangeparametersbetweentwo link peers.

Failover

ITadministratorstypicallyusefailoversolutionssuppliedbytheirstorageOEMsorthoseintegratedintotheOSplatform.Theirimplementationandmanagementmayalsobedifferent inFibreChannelandiSCSIenvironments.ForMicrosoftWindowsenvironments,somenetworkinterfacecard(NIC)vendorsprovideaNIC teaming driver that provides failover capabilities. It is expected that thiscapabilitymayalsobemadeavailablethroughtheOSplatform.


Chapter 6: Performance

SAN performance and capacity managementSANperformancecanbeadverselyaffectedwhenstorageresourcesare lowor become constrained. This can cause application performance problems andservice level issues.Many ITorganizationsattempt to avert such issuesbyoverpurchasingandoverprovisioningstorage.Howeverthismethodologyfrequently results in wasted capital since the additional storage investment may notnecessarilybefullyutilized.Analternativeapproachisperformanceandcapacityplanningpracticestoavoidunexpectedstoragecostsanddisruptiveupgrades. The objective is to predict storage needs over time and then budget capital and labor to make regular improvements to the storage infrastructure.

Inpractice,SANperformanceandcapacityplanningcanbequitechallengingaspredicting the storage needs of an application or department over time without a careful assessment of past growth and a comprehensive evaluation of future plansisvirtually impossible.Manyorganizationstendtoforegotheexpenseandeffortofa formalizedprocessunlessamission-criticalprojectorseriousperformanceproblemrequireit.Organizationschoosingtosustainanongoingperformance and capacity planning effort will need either comprehensive storage resourcemanagement(SRM)-typetooloracapacityplanningapplication.

With regards to performance monitoring and tuning tools, there are various benchmarkingtoolsavailable.Belowarejustsomeexamples.

Effect of Converged NetworkConverged networking will impact a data center’s performance processes,where today there are more questions than answers.

1.Howwilltrafficbesegregatedona10GbEpipesothatyoucanallocatebandwidthforstorageandnetworktraffic?

2.Whatmonitoringtoolswilltrackutilization?Currently,youindependentlymonitorloadsontheEthernetandFibreChannelcables.So,inconvergedenvironments,howdoyoudothis?

3.SpecifictoUniversalConvergedNetworkAdapters(UCNAs),iftheHBAisconfiguredasFCoE,canIalsorunsoftwareiSCSIoffit?WillTOEcapabilitiesbeavailable?

4.Howwillmultipathingconfigurationsbedeployed?Wecurrentlyhave:

i.IPmultipathing(twoNICconnectedtotwoswitches)

ii.FibreChannelmultipathing


5.Willconvergedenvironmentshavespecialcablingrequirements?(e.g.,TYPE:CAT5,CAT6oranyspecialtypecables,distance)

6.HowdoyouimplementandmonitorQoS?Hardware-basednetworkanalyzersatthenetworklevelneedtosupportconvergednetworkstomonitortrafficutilization.Inconvergedenvironments,howcantheanalyzerstellapartthetrafficonasinglephysicalcable?

Industry Benchmarks

Storage Performance Council (SPC)

SPC Benchmark 1: Consistsofasingleworkloaddesignedtodemonstratethe performance of a storage subsystem while performing the typical functions of business critical applications. Those applications are characterized bypredominatelyrandomI/Ooperationsandrequirebothqueriesaswellasupdateoperations. Examples of those typesof applications includeOLTP, databaseoperations, and mail server implementations.

SPC Benchmark 2: SPC-2consistsof threedistinctworkloadsdesignedtodemonstratetheperformanceofastoragesubsystemduringtheexecutionofbusiness critical applications that require the large-scale, sequential movement of data. Those applications are characterized predominately by large I/Osorganized into one ormore concurrent sequential patterns. A description ofeach of the three SPC-2 workloads is listed below as well as examples ofapplicationscharacterizedbyeachworkload.

•LargeFileProcessing:Applicationsinawiderangeoffields,whichrequiresimplesequentialprocessofoneormorelargefilessuchasscientificcomputingandlarge-scalefinancialprocessing.

•LargeDatabaseQueries:Applicationsthatinvolvescansorjoinsoflargerelational tables, such as those performed for data mining or business intelligence.

•VideoonDemand:Applicationsthatprovideindividualizedvideoentertainmenttoacommunityofsubscribersbydrawingfromadigitalfilmlibrary.

FormoreinformationonStoragePerformanceCouncilbenchmarks,pleasevisitwww.storageperformance.org

Transaction Processing Performance Council (TPC)

TPC-C:Simulatesacompletecomputingenvironmentwhereapopulationofusersexecutes transactionsagainst adatabase.Thebenchmark iscenteredaround the principal activities (transactions) of an order-entry environment.These transactions include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses. Whilethebenchmarkportraystheactivityofawholesalesupplier,TPC-Cisnot

http://www.storageperformance.org


limited to the activity of any particular business segment, but, rather represents any industry that must manage, sell, or distribute a product or service.

TPC-C involves a mix of five concurrent transactions of different types andcomplexity either executedon-lineorqueued for deferred execution. It doesso by exercising a breadth of system components associated with suchenvironments,whicharecharacterizedby:

•Thesimultaneousexecutionofmultipletransactiontypesthatspanabreadthofcomplexity

•On-lineanddeferredtransactionexecutionmodes

•Multipleon-lineterminalsessions

•Moderatesystemandapplicationexecutiontime

•Significantdiskinput/output

•Transactionintegrity(ACIDproperties)

•Non-uniformdistributionofdataaccessthroughprimaryandsecondarykeys

•Databasesconsistingofmanytableswithawidevarietyofsizes,attributes,and relationships

•Contentionondataaccessandupdate

•TPC-Cperformanceismeasuredinnew-ordertransactionsperminute.Theprimarymetricsarethetransactionrate(tpmC),theassociatedpricepertransaction($/tpmC),andtheavailabilitydateofthepricedconfiguration.

TPC-E:

TPCBenchmark™E(TPC-E) isanewOn-LineTransactionProcessing(OLTP)workloaddevelopedby theTPC.TheTPC-Ebenchmarkusesadatabase tomodel a brokerage firm with customers who generate transactions relatedto trades,account inquiries,andmarket research.Thebrokeragefirm in turninteractswithfinancialmarkets toexecuteordersonbehalfof thecustomersand updates relevant account information.

Thebenchmark is“scalable,”meaning that thenumberofcustomersdefinedfor thebrokerage firmcanbe varied to represent theworkloadsofdifferent-sizebusinesses.Thebenchmarkdefines the requiredmixof transactions thebenchmarkmustmaintain.TheTPC-Emetricisgivenintransactionspersecond(tps).ItspecificallyreferstothenumberofTrade-Resulttransactionstheservercan sustain over a period of time.

Although the underlying business model of TPC-E is a brokerage firm, thedatabase schema, data population, transactions, and implementation rules havebeendesignedtobebroadlyrepresentativeofmodernOLTPsystems.


Benchmarking Software

Iometer

IometerisanI/Osubsystemmeasurementandcharacterizationtoolforsingleandclusteredsystems.Itisusedasabenchmarkandtroubleshootingtoolandiseasilyconfiguredtoreplicatethebehaviorofmanypopularapplications.OnecommonlyquotedmeasurementprovidedbythetoolisI/Opersecond(IOPs).

Iometerisoneofthemostpopulartoolamongstoragevendorsandisavailablefree from www.iometer.org

IOzone

IOzoneisafilesystembenchmarktool.Thebenchmarkgeneratesandmeasuresavarietyoffileoperations.Iozonehasbeenportedtomanymachinesandrunsundermanyoperatingsystems, performingabroadfilesystemanalysisofavendor’s computer platform.

IOzoneisavailablefreefromwww.iozone.org

While running benchmarks, care should be taken avoid the following common mistakes:

•Testingstorageperformancewithfilecopycommands

•Comparingstoragedevicesback-to-backw/oclearingservercache

•Testingwherethedatasetissosmallthebenchmarkrarelygoesbeyondserver to storage cache

•Forgettingtomonitorprocessorutilizationduringtesting

•Monitoringthewrongserver’sperformance

This will ensure a more realistic and representative assessment of your environment.

Ixia IxChariot

IxChariot is a fee based benchmarking tool which simulates applicationsworkloads to predict device and system performance under realistic load conditions.IxChariotperformsthoroughnetworkperformanceassessmentanddevice testing by simulating hundreds of protocols across thousands of network endpoints.

When vendors utilize such benchmarking tools to asses performance, theytake into consideration the entire network, as the server, network and storage systemallplayapartinapplicationperformance.It’simportanttounderstandhow to identify and eliminate latency bottlenecks to ensure superior application performance. While it may be logical to look for sources of performance degradation outside the server – in the network connectivity or storagecomponents – it’s important to understand that performance degradation

http://www.iometer.org

http://www.iozone.org


canalsooccurwithin theserver.Forexample thecycles theserverCPUhasavailable to process application workloads can impact performance. This is referredtoasaserver’sCPUefficiency.WhataffectsCPUefficiencyisfurtherdiscussed below.

Therefore a properly designed SAN can improve storage utilization, highavailability and data protection.

WhenevaluatingSANperformance,thefollowingneedtobeconsidered:

• Latency

• Bandwidth

• Throughput

• Input/Outputoperationspersecond(IOPS)

Fibre Channel has evolved over the years, delivering faster and fasterperformance, as measured by throughput (megabits per second). Today;however, 10Gb based Ethernet networks now provide performance equal to FibreChannelbasednetworks.10GbEiscurrentlythefastestof theEthernetstandards, with a nominal data rate of 10Gb/s or 10 times as fast as Gigabit Ethernet.ThefollowingtableprovidesaperformancesummaryofFibreChannelevolution, along with 10GbE for comparison.

NameThroughput

(MBps)*Line-Rate –

1GbFibreChannel 200MB/s 1.0625GBaud –




16bFibreChannel 3200MB/s 17.00GBaud –

1Gig bit Ethernet 1Gb second 1Gigabit / sec.

10Gig bit Ethernet 10Gb second 10Gigabits / sec.

40Gig bit Ethernet 40Gb second 40Gigabits / sec.

* -Throughputforduplexconnections

Key TerminologyThefollowingterminologyisimportanttounderstandingSANperformance:

CPU Efficiency

CPU efficiency has various definitions. In context of this document, CPU


efficiency is referring to the serverprocessor’s ability toprocessapplicationworkloads-orsimplyputapplicationworkload IOPrequirementsdividedbytheserver’sCPUspeed(GHz).ThemoreIOPSthatcanbeprocessedbyeachGHz,thehighertheCPU’sefficiency.Afactorthatcanimpactaserver’sCPUefficiency is theHBAselection.SomeHBAsoff-loadcertainprocessesontothe server’s processer. By doing so, the server processor has less cyclesavailable for application workload processing, which can in turn lowers network performance. Therefore, proper HBA selection can be one of the simplestmethodsof improvingoverallperformance.CPUefficiencyalsoaffordsotherbenefits,includingreductionofcapitalandoperationalexpenditures.

Performance TuningStoragesystemsrelyonanumberofperformancetuningprocessesdescribedbelow.

Driver Parameters

Anotherfactorthatcanimpactperformanceisthedriverparameter(alsoknownas adapter parameter) settings. The optimumsettings are either dynamicallymanagedbythedriverorconfiguredautomaticallyduringtheadapterinstallationusing the adapter’s management application.

Queue depth setting

Queuing refers to the ability of a storage system to queue storage commands for later processing. Queuing can take place at various points in your storage environment, from the Host Bus Adapter (HBA) to the storage processor/controller. Forexample,modifying the “HBAQueueDepth” isaperformancetuningtipforserversthatareconnectedtoSANs.SincetheHBAisthestorageequivalent of a network card, the Queue Depth parameter controls how much data is allowed to be “in flight” on the storage network from that card. Most cards default to a queue depth of 32, which is perfect for a general purpose server andpreventstheSANfromgettingtoobusy.Queuedepthcanbeadjustable.Note that a little queuing may be acceptable depending on the transactionworkload,buttoomanyoutstandingI/Oscannegativelyimpactperformance,as measured in latency.

Interrupt coalescing

Interrupt coalescingbatches up kernel interrupts from theNIC to the kernel,reducing per packet overhead. Interrupt coalescing represents a trade-offbetweenlatencyandthroughput.Coalescinginterruptsalwaysaddslatencytoarrivingmessages,buttheresultingefficiencygainsmaybedesirablewherehighthroughput is desired over low latency. Troubleshooting latency problems often pointtointerruptcoalescinginGigabitEthernetNIChardware.Fortunately,thebehaviorofinterruptcoalescingisconfigurableandcangenerallybeadjustedto


theparticularneedsofanapplication.ThedefaultforsomeNICsordriversisan“adaptive”or“dynamic”interruptcoalescingsettingthatseemstosignificantlyfavor high throughput over low latency. The details of configuring interruptcoalescingbehaviorwillvarydependingontheOSandperhapseventhetypeofNICinuse.

Key MetricsThefollowingarekeySANperformancemetrics:

Latency:I/Olatency,alsoknownasI/Oresponsetime,measureshowfastanI/OrequestcanbeprocessedbythediskI/Osubsystem.ForagivenI/Opath,itisinproportiontothesizeoftheI/Orequest.Thatis,alargerI/Orequesttakeslonger to complete.

Bandwidth:Theamountofavailableend-to-endSANbandwidthisdependenton back-end storage capacity on the SAN side. Improving SAN bandwidthrequiresconsiderationofsuchfactorsashowthestorageisconfigured,whattheapplicationworkloadisandwhereacurrentbottleneckexists.Forexample,ifeachserveraccessesaseparateuniqueLUN,addingasecondHBAwouldadd more bandwidth, but you might not see a performance improvement. This wouldbethecaseiftheLUNisbeingaccessedviaasingleadapterpathaswellasiftheadapterortheLUNarenotthebottlenecks.OrconsiderifeachserveraccessesmultipleLUNs;iftheLUNsareloadbalancedacrossadapters,thereis the potential for performance improvement.

Throughput: Throughput measures how much data can be pumped through thediskI/Opath.IfyouviewtheI/Opathasapipeline,throughputmeasureshowbigthepipelineisandhowmuchpressureitcansustain.So,thebiggerthe pipeline is and the more pressure it can handle, the more data it can push through.ForagivenI/Opath,throughputisindirectproportiontothesizeoftheI/Orequests.Thatis,thelargertheI/Orequests,thehigherthemegabytespersecond(MBps).LargerI/OsgiveyoubetterthroughputbecausetheyincurlessdiskseektimepenaltythansmallerI/Os.

IOPS:I/OOperationsPerSecond(alsoknownasIOPS)isameasureofadeviceoranetworkabilitytosendandreceivepiecesofdata.Thesizeforthesepricesofdatadependsontheapplication(ie:transactional,database,etc.)andgenerallyrange in size from 512byte to 8kilo bytes. IOPS have a known performanceprofileofraisingCPUutilizationfromacombinationofCPUinterruptandwaittimes.ThespecificnumberofIOPSpossibleinanyserverconfigurationwillvarygreatly depending upon the variables entered into the program, including the balanceofreadandwriteoperations,themixofrandomorsequentialaccesspatterns and the number of worker threads and queue depth, as well as the datablocksizes.

Transfer Rate: Transfer rate is the amount of data that can be transferred on a specifictechnology(ie:2Gb,4Gbor8GbFibreChannel)withinaspecifictimeperiod.Instoragerelatedtests,thetransferrateisexpressedinmegabytesor


gigabytespersecond;MB/sandGB/srespectively.Highsustainable transferrate play a critical in applications which “stream” data. These include backup andrestore,continuousdataprotection,RAID,videostreaming,filecopyanddata duplication applications.

CPU Efficiency (based on IOPS): This metric examines the ratio of IOPSdivided by average CPU utilization. This ratio illustrates the efficiency of agiventechnologyintermsofCPUutilization.HighernumbersofCPUefficiencyshow that the given technology is friendlier to the host system’s processors. Higher bandwidth or IOPS with lower CPU utilization is the desired result.Thisisimportant,asusersaretryingtomaximizetheir investments,andCPUutilization.

IOPSThemostcommonperformancecharacteristicsthataremeasuredordefinedare:

•TotalIOPS:TotalnumberofI/Ooperationspersecond(whenperformingamixofreadandwritetests)

•RandomReadIOPS:AveragenumberofrandomreadI/Ooperationspersecond

•RandomWriteIOPS:AveragenumberofrandomwriteI/Ooperationspersecond

•SequentialReadIOPS:AveragenumberofsequentialreadI/Ooperationsper second

•SequentialWriteIOPS:AveragenumberofsequentialwriteI/Ooperationsper second

LatencySANscannottoleratedelay.Theperformanceofstoragenetworksisextremelysensitive todata/frame loss.WhileLANtraffic is lesssensitive,slowingdownaccesstostoragehasasignificantimpactonserverandapplicationperformance.Inaddition,suchdelaysalsonegativelyimpactserver-to-servertraffic.Forthatreason, Fibre Channel has been the network protocol of choice for storagenetworking, providing high-performance connectivity between servers and their storageresources.FibreChannel isanexampleofa loss-lessnetwork in thesensethatadatatransmissionfromthesender(initiator/server)isonlyallowediftherecipient(target/storagearray)hassufficientbuffer(memory)toreceivethedata. This ensures data is not “dropped” by the recipient.


Chapter 7: Security

Due to compliance or risk concerns, storage administrators must be aware of the accessibility and vulnerabilities that storage systems are exposed to vianetwork interconnections. Protecting sensitive data residing in and flowingthrough storage networks should be part of risk management assessments. Defense in depth approaches to security include applying solutions that balance the risks and costs with the desire to apply best practices for securing storage systems.

Securitycontrols,whethertheyarepreventive,detective,deterrent,orcorrectivemeasures can be categorized into physical, procedural, technical, or legal/regulatory compliance controls. There are several documents that promote goodsecuritypracticesanddefine frameworks to structure theanalysis anddesign for managing information security controls. These include documents fromISO(27001/2)andNIST.SNIApublishesbestpracticesforstoragesystemsecurity.

Technical solutions are available to implement controls of the confidentiality,integrity,andavailabilityofinformation.Inadditionconcernsaboutaccountabilityandnon-repudiationshouldbeconsidered.Accesscontrolsandauthorizationcontrolscanpreventaccidentsandrestrictprivileges.Authenticationofusersand devices can provide network access controls. Protecting managementinterfaces, including replacement of default passwords, assures protection fromunauthorizedchanges.Auditandloggingsupportprovidesforsupportforvalidationofsecurityconfigurationsandsupportfororganizations’policies.

Security in Converged Networking EnvironmentsMany IT organizations are acknowledging the benefits and advantages ofconverged networking environments, primarily the sharing of infrastructure and the reduction of costs. Network convergence allows unprecedentedconnectivity options to information via platforms that are capable of supporting blockstoragetrafficsuchasiSCSI,FC,andFCoE,aswellasfileservicetrafficforNAS(NFS/CIFS/SMB2)storage.Asnetworksandstorageincreasinglysharethesameinfrastructures,thesecurityaspectssuchasconfidentiality,integrity,and availability are to be considered in risk assessments. Authentication,confidentiality, user ID andcredentialmanagement, audit support, andothersolutions relevant to converged or virtualized traffic flows can provide newopportunitiesforefficiencybyconsideringcommonsecuritysolutionswheneverpossible. Many customers are finding that protocol agnostic and storageagnostic solutions will prove to be economical solutions to assist them in meeting security and compliance requirements.


Security BreachesThe inherent architecture of Fibre Channel SAN affords it greater degree ofsecurity.HoweverthisisnottosayaSANisimpervioustosecuritybreaches.Commonrisksinclude:

•Compromisedmanagementpathcanoccurwhentheorganizationhasa:

- Mal-intentioned administrators

-Compromisedmanagementconsole

-Unsecuremanagementinterfaces

To avoid such situations, organizations typically implement managementauthorizationandaccesscontrolaccessprocessesaswellasauthenticationmeasures. Therefore it is critical to select components that support role based policies and authentication features.

•Unauthorizeddataaccess

-ThistypicallyoccurswhenastorageLUNbecomesaccessiblebeyondtheauthorizedhosts.Theimplicationofsuchaneventmeansthatpeople who should not have access to certain data will now be able to accessit.LUNmasking/mapping,typicallydoneatthearraylevel,ishow such conditions are addressed.

•Impersonationsandidentityspoofing

- This condition occurs when initiators fake their identity through worldwidename(WWN)spoofingwhichenableasessiontobehijacked.ToprotectagainstsuchoccurrencesintheSAN,organizationsleverageDH-CHAP,atypeofauthenticationandIKE,which establishes shared security information between two network entities to support secure communication.

ApplyingtightercontrolstooverallSANconfigurationsisalsohelpfulasitwouldprevent administrative errors which could leave a SAN vulnerable to suchattacks.

•Compromisedcommunication

-Thiscanbeoneofthemostcostliestbreachesforanorganization.Notonlyarethereregulatoryimplications,intermsoffines,butalsobusiness implications, in terms of loss of intellectual property and lossofcustomerconfidence.Thereforegreatcaremustbetakentoprotectdatafrominterceptionoreavesdropping.Lossofdataintegrityis another way “communication” can be compromised as data can be intercepted,modifiedandthensentonitsway.

Therefore great care must be taken to prevent compromised communicationandlossofdataintegrity.Organizationshouldleveragedataencryptiontoencrypttheirdata.Althoughtherearevarious encryption methodologies, host based encryption is the most


effective as it encrypts data at source of its origin, protecting the data in flight and at rest. Even in case of a lost or stolen hard disk drive, thedataremainsencrypted.Toreducedataintegrityincidents,SANadministrators are showing greater interest in products from vendors whosupportindustryinitiativessuchasDataIntegrityInitiative(DII),which provides application to disk data integrity protection.

Methods of Protecting a SANThe following are methods storage administrators leverage to augment security withinSANs.

Zoning

Fabric Zoning

The zoning service within a Fibre Channel fabric was designed to providesecurity between devices sharing the same fabric. The primary goal was to prevent certain devices from accessing other devices within the fabric. With many different types of servers and storage devices on the network, the need for security is critical. For example, if a host were to gain access to a diskbeingusedbyanotherhost,potentiallywithadifferentOS,thedataonthisdiskcould become corrupted. To avoid any compromise of critical data within the SAN,zoningallowstheusertooverlayasecuritymapdictatingwhichdevices,namely hosts, can see which targets, thereby reducing the risk of data loss.

Zoning does, however, have its limitations. Zoning was designed to do nothing more than prevent devices from communicating with other unauthorizeddevices. It isadistributedservice that iscommonthroughout the fabric.Anyinstalledchangestoazoningconfigurationarethereforedisruptivetotheentireconnected fabric. Zoning also was not designed to address availability or scalability of a FibreChannel infrastructure. Therefore,while zoningprovidesanecessaryservicewithinafabric,theuseofVSANs,describedbelow,alongwithzoning,providesanoptimalsolution.

WWN Zoning

WWNzoningusesnameserversintheswitchestoeitheralloworblockaccesstoparticularWWNsinthefabric.AmajoradvantageofWWNzoningistheabilityto re-cable the fabricwithout having to redo the zone information.However,WWNzoningissusceptibletounauthorizedaccess,asazonecanbebypassedifanattackerisabletospooftheWWNofanauthorizedadapter.

SAN zoning

SANzoningisamethodofarrangingFibreChanneldevicesintologicalgroupsover the physical configuration of the fabric. SAN zoning can be used tocompartmentalizedata forsecuritypurposes.SANzoningalsoenableseachdeviceinaSANtobeplacedintomultiplezones.


Hard Zoning

Hard zoning occurs in hardware; therefore, the zone is physically isolated,blockingaccesstothezonefromanydeviceoutsideofthezone.

Soft Zoning

Softzoningoccursatthesoftwarelevel;thus,itismoreflexiblethanhardzoning,making rezoning processes easier. Soft zoning uses filtering implemented inFibreChannelswitchestopreventportsfrombeingseenfromoutsideoftheirassigned zones. It usesWWNs to assign security permissions. The securityvulnerability in soft zoning is that the ports are still accessible if the user inanotherzonecorrectlyguessestheFibreChanneladdress.

Port Zoning

Port zoning uses physical ports to define security zones, enabling ITadministrators to control data access through port connections. With port zoning,zone informationmustbeupdatedevery timeauserchangesswitchports.Inaddition,portzoningdoesnotallowzonestooverlap.Portzoningisnormallyimplementedusinghardzoning,butcouldalsobeimplementedusingsoftzoning.

Virtual SAN

VSAN is aCisco technology, designed to enhance scalability and availabilitywithintheFibreChannelnetworks.Itaugmentsthesecurityservicesavailablethroughfabriczoning.VSANsenableITadministratorstotakeaphysicalSANandestablishmultipleVSANson topof it,creatingcompletely isolated fabrictopologies, eachwith its own set of fabric services. Since individual VSANspossesstheirownzoningservices,eachisindependentoftheotheranddoesnotaffectzoningservicesofotherVSANs.

SomebenefitsofVSANsinclude:

a.IncreasedutilizationofexistingassetsandreducedneedtobuildadditionalphysicallyisolatedSANs

b.ImprovedSANavailabilitybynotonlyprovidinghardware-basedisolation,butalsotheabilitytofullyreplicateasetofFibreChannelservicesforeachVSAN

c.GreaterflexibilitythroughselectiveadditionordeletionofVSANsfromatrunklink,controllingthepropagationofVSANsthroughthefabric

Asasidenote,VLANsallowstheextensionofaLANovertheWANinterface,overcoming the physical limitations of a regular LAN. Just as with VSANs,VLANs enable IT administrators to take a physical LAN and overlay on topmultipleVLANs.VLANtechnologyalsoallowsITadministratorstodeployseveralVLANsoverasingleswitchinsuchamannerthatalltheLANswilloperateasindependent networks.


LUN Masking

LUNmaskingisanauthorizationprocessthatmakesaLUNavailabletosomehosts andunavailable to other hosts. LUNmasking is implementedprimarilyat theHBA level.LUNmasking implementedat this level isvulnerable toanyattackthatcompromisestheHBA.SomestoragecontrollersalsosupportLUNmasking. An additional benefit to LUNmasking is that it preventsWindowsoperatingsystemstowritevolumelabelsonallavailable/visibleLUNswithinthenetwork,whichcanrendertheLUNsunusablebyotheroperatingsystemsorresult in data loss.

Security Protocols

Fibre Channel Authentication Protocol

Fibre Channel Authentication Protocol (FCAP) is an optional authenticationmechanism used between any two devices or entities on a Fibre Channelnetworkusingcertificatesoroptionalkeys.

Fibre Channel Password Authentication Protocol

Fibre Channel Password Authentication Protocol (FCPAP) is an optionalpassword-based authentication and key exchange protocol that is utilized inFibreChannelnetworks.FCPAPisusedtomutuallyauthenticateFibreChannelportstoeachother.ThisincludesE_Ports,N_PortsandDomainControllers.

Switch Link Authentication Protocol

Switch Link Authentication Protocol (SLAP) was designed to prevent theunauthorized addition of switches into a Fibre Channel network. It is anauthenticationmethodforFibreChannelswitchesthatusesdigitalcertificatesto authenticate switch ports.

Fibre Channel - Security Protocol

FibreChannel-SecurityProtocol(FC-SP)isasecurityprotocolforFibreChannelProtocol (FCP) and fiber connectivity (Ficon). FC-SP is aprojectof TechnicalCommittee T11 of the InterNational Committee for Information TechnologyStandards (INCITS).FC-SP isasecurity frameworkthat includesprotocolstoenhanceFibreChannelsecurityinseveralareas,includingauthenticationofFibreChanneldevices,cryptographicallysecurekeyexchangeandcryptographicallysecurecommunicationbetweenFibreChanneldevices.FC-SP is focusedonprotectingdata in transit throughout theFibreChannelnetwork.FC-SPdoesnotaddressthesecurityofdatastoredontheFibreChannelnetwork.

Diffie Hellman - Challenge Handshake Authentication Protocol

FC-SPdefinesDiffieHellman-ChallengeHandshakeAuthenticationProtocol(DH-CHAP)asthebaselineauthenticationscheme.DH-CHAPpreventsWorld


WideName(WWN)spoofing(i.e.,impersonation,masqueradingattacks)andisdesigned to withstand replay, offline dictionary password lookup and challenge reflection attacks. (See Figure 8 for an illustration of the threats preventedthroughtheimplementationofDH-CHAPauthenticationbytheHBA/CNA.)DH-CHAPsupportsalgorithm-basedauthenticationsuchasMD-5andSHA-1.

Figure 8:HostthreatspreventedbyimplementationofDH-CHAPauthenticationbytheHBAorUCNA.

Encapsulating Security Payload over Fibre Channel

Encapsulating Security Payload (ESP) is an Internet standard for theauthentication and encryption of IP packets. ESP is widely deployed in IPnetworksandhasbeenadaptedforuseinFibreChannelnetworks.TheInternetEngineeringTaskForce(IETF)iSCSIproposalspecifiesESPlinkauthenticationandoptionalencryption.ESPoverFibreChannelisfocusedonprotectingdataintransitthroughouttheFibreChannelnetwork.ESPoverFibreChanneldoesnotaddressthesecurityofdatastoredontheFibreChannelnetwork.

Securing iSCSI, iFCP and FCIP over IP Networks

TheIETFIPStorage(IPS)WorkingGroupisresponsiblefordefiningstandardsfortheencapsulationandtransportofFibreChannelandSCSIprotocolsoverIPnetworks.TheIPSWorkingGroup’scharter includesresponsibilityfordatasecurity, security including authentication, keyed cryptographic data integrity andconfidentiality,sufficienttodefendagainstthreatsuptoandincludingthosethat canbe expected on a public network. Implementation of basic securityfunctionalitywillberequired,althoughusagemaybeoptional.TheIPSWorkingGroupdefines theuseof theexisting IPsecand InternetKeyExchange (IKE)protocolstosecureblockstorageprotocolsoverIP.


Effect of Converged NetworkGiventheunifiednaturewithinaconvergedenvironment,precautionshavetobe put in place to address access control, preventing the network administrator undoing something the server administrator did. Currently SAN, server andnetworkadministrationareindependentofeachother;however, inconvergedenvironments, management of these areas will overlap.

Native FCoE Storage

Storage arrays supporting native FCoE interfaces will enable end-to-endnetwork convergence and are expected to be the next logical progressionin theconvergednetworkenvironment.Besides thechange inphysical layerconnectivity that encapsulates Fibre Channel frames over Ethernet, thefunctionalityprovidedbynativeFCoEarraysremainsequivalenttothatofaFibreChannelarray.ThenativeFCoEarrayswillleveragetheprovenperformanceofFibreChannelstackandretaintheexistingprocessesrequiredforLUNmaskingandstoragebackup(seeFigure9).

Zoning

ZoningpracticesusedinFibreChannelnetworkingtypicallyremainunaffectedinaconvergednetworkenvironment.ProcessesaretransparentlycarriedovertotheFCoE-capablelosslessEthernetswitch.


Figure 9:AnativeFCoEstorageconnectedtoFCoE-enablednetwork

LUN Masking

LUNmasking practices used by the storage administrators in FibreChannelstorageremainunaffectedinaconvergednetworkenvironment.ProcessesaretransparentlycarriedovertonativeFCoEstorage.

Compliance

Internal business initiatives and external regulations are constantly adding tocompliance challenges and are testing the capabilities of status quo networks. AlthoughITmanagerscouldcontinuetodeploymultiplenetworksandensurecompliance, the process gets tedious with the changing dynamics of SANexpansiondrivenby virtual serversandbladeservers.Asimplifiedapproachto networking provides competitive advantages in the face of new business initiatives and helps meet regulatory compliance obligations.


Chapter 8: Management: Configuration and Diagnostics

Networkadministratorsareconcernedwithmovementofdata,ortobemorespecific, the reliable of user data from one point to another point within thenetwork. Therefore the network administrator is interested in factors that affect management. Examples of such factors include bandwidth utilization,provisioning of redundant links to ensure secondary data paths, support for multiple protocols and so forth.

Storage administrators on the other hand, are less concerned about datatransportthanabouttheorganizationandplacementofdataonceitarrivesatitsdestination.LUNmapping,RAID levels,file integrity,databackup,storageutilization and so forth comprise the bulk of a storage administrator’s dailymanagement routines.

These different views of management converge in a SAN, since the properoperationofaSANrequiresbothmanagementofdatatransportandmanagementofdataplacement.Byintroducingnetworkingbetweenserversandstorage,aSAN forces traditional storagemanagement to broaden its scope to includenetwork administration and encourages traditional network management to extenditsreachtodataplacementandorganization.SomeofthemostfrequentquestionsSANadministratorsneedtoanswerare:

• HowmuchstoragedoIhaveavailableformyapplications?

• Whichapplications,usersanddatabasesaretheprimaryconsumersofstorage?

• WhendoIneedtobuymorestorage?

• Howisstoragebeingused?

SAN’sstorageresourcescanbemanagedcentrally,allowingadministratorstoorganize,provisionandallocatethatstoragetousersorapplicationsoperatingonthenetworkacrossanorganization.Centralizationalsoallowsadministratorsto monitor performance, troubleshoot problems and manage the demands of storage growth.

SAN provisioningTo centralize storage on a SANwhile restricting access to authorized usersor applications; the entire storage environment should not be accessible toevery user. Administrators must carve up the storage space into segments


thatareonlyaccessibletospecificusers.Thismanagementprocessisknownas provisioning. For example, some amount of data center storage may beprovisioned for a purchasing related application that may only be accessible by the purchasing department, while other space may be apportioned for personnel records accessible only to the human resources department.

The major challenge with provisioning relates to storage utilization. Oncespace is allocated, it cannot easily be changed. Thus, administrators typically provision ample space for an application’s future use.Unfortunately, storagecapacity that is provisioned for one application cannot be used by another, so space that is allocated, but unused, is basically wasted until called for by the application.Thisneedtoallocateforfutureexpansionoftenleadstosignificantstoragewasteonthestorageareanetwork.Onewaytoalleviatethisproblemisthrough thin provisioning, which essentially allows an administrator to “tell” an application that some amount of storage is available but actually commit far less drivespace—expandingthatstoragein later incrementsastheapplication’sneeds increase.

Provisioningisaccomplishedthroughtheuseofsoftwaretools.Toolstypicallyaccompany major storage products. The issue for administrators is to seek a provisioning tool that offers heterogeneous support supporting the storage platforms currently in their datacenter.

CreatingaSANinvolvesmorethansimplycablingserversandstoragesystemstogether. Resources must be configured, allocated, tested and maintained.IntroductionofnewdevicestotheSANcanchangetherequirements,thereforemanagement is a key consideration and it’s important to select solutions that canminimizethetimeandeffortneededtokeepaSANrunning.

Manageabilityhasasignificantimpactondatacenters.Streamliningdeployment,installationandconfigurationprocessestoimproveefficiencyarecriticalforITorganizationsthatarechallengedwithservicingincreasingbusinessdemandswithshrinking resources.Anotherkeyaspectofmanagement is theability tomonitor,diagnoseandobtaininformationonthehealthoftheSAN.

It is important to understand that storage traffic does not tolerate data loss;therefore, it requires advanced management granularity. To that end, a more comprehensive set of tools have been developed to provide switch fabric, initiators, targets (storage arrays) and LUN administrative capabilities. Thisenables the storage network to be kept at an optimum level of performance.

Inaddition,likeEthernetnetworks,FibreChannel-basedSANshavearobustsetof error checking and diagnostic capabilities designed to ensure the highest level ofnetworkperformanceandconnectivity.Inaddition,thereareabroadrangeof tools that enable storage administrators to address any issues that may arise within their networks. These include diagnostic tools that help troubleshoot:

• Portfunctionality(initiatorandtarget)

- Adapterportlevel


- Storageportlevel

- LUNandspindle

- Switchport

• I/Odiagnostics

- PerformancefromIOPSperspective

- Performancefromlatencyperspective

- Error detection

Adapter ManagementAdaptermanagementcanbebrokendowninthefollowingmanner:

Installation

This entails the physical installation of the adapter within the server as well as the adapter’s software components. It is important to select adapterswhichprovidegreatestinstallationflexibilityassuchcapabilitiescansignificantlyhelptostreamlinedeployments,improveserveravailabilityandreducecosts.Someexamplesofsuchcapabilitiesincludetheabilitytopre-configureaserverwiththe adapter’s software without the adapter being present in the server. This helps topre-stageserverresourcesforrapiddeployment. Installationautomation isalsoanotherfeaturewhichshouldbetakenintoconsideration.Automationcanspeedup and streamline adapter installation by deploying software components in a “batch” fashion.

Configuration

Once the HBA has been installed, it must be configured. Using the HBA’smanagementapplication,SANadministratorssetthe“driverparameter”settingstocustomize theHBAscapabilities tomatch theneedsof their environment.There is a host of setting which administrators can set to activate features and change performance characteristics of the adapter. Some examples includequeue depth settings for optimal operation with existing storage resources,securitysettings,virtualization,timeouts,etc.BootfromSANsettingscanalsobe set during the configuration process. As server vendors shift to disklessserverdesigns,abootdevicemustthenbeassignedtotheserverfromtheSAN.Suchserverscanalsobeassignedwithasecondarybootdevice,incasetheprimarybootdevicebecomeinaccessible.Certainadaptervendorsalsoprovideconfiguration automation capabilities aswell, enablingSANadministrators tostreamlinemanagementcapabilities.Anexampleofconfigurationautomationistheabilitytocentrallypropagateadapterfirmwareanddriverupdatesacrosstheentirenetwork,helpingtoreduceserverre-boots,maximizenetworkuptimeandincreaseoverallmanagementflexibility.


Management

Adaptermanagementshouldbeacriticalconsiderationinselectinganadapterfortheserver.ITadministratorsingeneralaretaskedtodomorewithsamelevelof resources. To that end, they need management tools which help them improve administration of adapterswithin the data center. Convergence introduces anew layer ofmanagement requirements. Fibre Channel adapter vendors arenow also offering FCoE, NIC and iSCSI solutions as well. However, somevendors have yet to integrate central management of their server to network connectivity solutions. That is why it is important to select adapter vendors which provide a centralized, cross platformmanagement solution for unifiedadministration of adapters, regardless of the protocol (Fibre Channel, FCoE,iSCSIorNIC).SuchsolutionscancentrallydisplayalladapterswithinaSAN,enablingeffectiveandefficientmanagementofadapters.Byselectingtherightadapter,SANadministratorscansimplifyadministrativetasksandimprovedatacenter responsiveness support demands of dynamic business environments.

Diagnostics

Given the critical nature of a SAN, robust set of diagnostics are amust forthe various pieces that comprise a SAN, which includes the adapter. Whilethere are a common set of diagnostic tools offered by adapter vendors, some vendors have developed advanced set of diagnostic and I/O Managementapplicationsdesignedtotrulyoptimizenetworkavailability,assetutilizationandresponsiveness.Such tools canbeused to identify andaddress intermittentSAN issues, over subscription conditions, and end-to-end I/O performancedegradations.

Key TerminologyThefollowingsectiondefinessomecommontermsandmanagementfunctionsused by storage administrators.

HBA and CNA configuration

RelativetoIPnetworks,thereismoreinvolvedinconfiguringconnectivity(HBAsand CNAs) for storage networks (relative to NICs used in IP networks). Forexample,whenconfiguringstorageadapters,storageadministratorsneedto:

• Knowhowtoplanandprovisionstorageresources

• Allocatestorageresourcesbasedonuserrequirements,whichrequiresunderstandingoftheuser’srequirements(capacityneeded,performancerequired,availability,etc.)

• TuneadapterandstoragefabrictomatchtheoptimumI/Otransactionalcapabilities of the storage arrays


Port Configuration

Initially,youhavetomakesuretheport’sworldwideportname(WWPN)ispartofastoragenetworkzone.ThisensurestheservercanaccessthestorageontheSANfabric.

Boot from SAN

SimilartoPXEbootinIPnetworks,FibreChannelnetworksalsosupportbootingoftheserverfromanon-localharddisk.Thisiscalled“bootfromSAN.”WhileEthernetnetworksrequireahostofintermediary(DHCP,PXE,alongwithanFTPorHTTP)services,FibreChanneldoesnothavesucharequirement. InFibreChannelnetworks,theserverhasdirectaccesstothehighlyavailablestoragedeviceswithintheSAN,whichitcanuseforbooting.EnablingbootfromSANrequiresconfiguringthestoragedevice,suchastheHBA,withthebootimageandbootdiskinformationandtheninstallingtheOS.

vPorts

Similar to creating virtual end-points in Ethernet environments, storageadministratorscancreateFibreChannelvPorts.UsingN_PortIDVirtualization(NPIV), multiple vPorts can be assigned to one physical port. NPIV allowseachvPorttohaveitsownWWPN,auniqueidentifier.Storageadministratorsuse vPorts to apply SAN best practices, such as zoning, in virtual serverenvironments.

SMI-S

Storage Management Initiative Specification (SMI-S) defines DistributedManagementTaskForce(DMTF)managementprofilesforstoragesystems.Aprofiledescribesthebehaviorcharacteristicsofanautonomous,self-containedmanagement domain.SMI-S includesprofiles for adapters, arrays, switches,storagevirtualizer,volumemanagementandmanyotherdomains.A“provider”isanimplementationforaspecificprofile.

Ataverybasiclevel,SMI-Sentitiesaredividedintotwocategories:

• Clientsaremanagementsoftwareapplicationsthatcanresidevirtuallyanywherewithinanetworkprovidedtheyhaveaphysicallink(eitherwithinthedatapathoroutsidethedatapath)toproviders.

• Serversarethedevicesundermanagementwithinthestoragefabric.

Clients can be host-based management applications (storage resourcemanagement,orSRM),enterprisemanagementapplicationsorSANappliance-basedmanagement applications (e.g., virtualizationengines).Serverscanbedisk arrays, host bus adapters, switches, tape drives, etc.

By leveraging SMI-S, vendors offer open, standards-based interfaces andsolutions (hardware or software), enabling easier integration, interoperabilityand management.


CIM

TheCommon InformationModel (CIM) is an open standard, and part of theDMTF standard, that defines how managed elements in an IT environmentare represented as a common set of objects and relationships between them. This is intended to allow consistent management of these managed elements, independentoftheirmanufacturerorSMI-Sprovider.ItisalsothebasisfortheSMI-Sstandardforstoragemanagement.

Effect of Converged NetworkCurrently, storage administrators have a distinct set of diagnostic tools andprocesses for fault isolation and diagnoses of issues within Fibre Channelnetworks. Given that there will be a common infrastructure in converged environments, fault isolation procedures must be adjusted or changed to determine the best method to effectively identify and resolve issues within the converged network. For example, determining if Fibre Channel end-device(storage)canbeaccessed,itsresponsetime,etc.

Otherimpactsincludethefollowing:

• Administratorsneedtoconfigure10GbEDCBportstocarryLANandstoragetraffic,aswellasallocatebandwidth.

• WhenrunningFibreChanneloriSCSIoverEthernet,bothhavedirectbooting capabilities.

• As10GbEDCBwillbeusedformulti-traffictypes,anyphysicaldisruptionwilladverselyaffectstorage,LANandanyotherformsofdatatraffic.

Fibre Channel Initialization Protocol (FIP)

FibreChannel Initialization Protocol (FIP) discovers all FibreChannel deviceswithin an Ethernet network. It is the FCoE “control” protocol responsiblefor establishing and maintaining Fibre Channel virtual links between FCoEdevices.

Port Configuration

Thefollowingdescribesnewportconfigurationprocesses.

FCoE Port Configuration Process:

TheFCoEportconfigurationmirrors thatofFibreChannelportconfiguration.Themajordifference,however,isthatbeforeportconfigurationcantakeplace,we need tomake sure there is aConvergedEthernet connection to an FCFprovider through an FCoE switch. FCF provider establishes a connectionbetweentheFCoEadapterandtheFCoEswitch.Whenthisisoperational,theadapterwilldiscoverthepresentedSANfabricandalltargetswillbevisibleontheFCoEswitch.


iSCSI Port Configuration Process

UsingtheiSCSIadapter’smanagementapplication,theadaptermustbegivenan iSCSI-qualifiedname(IQN).The IQNof the iSCSI targetdeviceandthe IPaddressof the targetportalmustalsobeavailable.With the iSCSIadapter’smanagement application, a connection to the target can then be initiated via thetarget’sIPaddress.


Chapter 9: Emulex Solutions

About Emulex

Emulex® creates enterprise-class products that intelligently connect storage, servers and networks, and is the leader in converged networking solutions for thedatacenter.ExpandingonitstraditionalFibreChannelsolutions,Emulex’sConnectivity Continuum architecture now provides intelligent networkingservices that transition today’s infrastructure into tomorrow’s unified networkecosystem. Through strategic collaboration and integrated partner solutions, Emulexprovidesitscustomerswithindustry-leadingbusinessvalue,operationalflexibilityandstrategicadvantage.

Emulex Server-to-network Connectivity Solutions

Emulex designs and offers a broad range of server-to-network connectivitysolutions,qualifiedforusewithofferingsfrommajorserverandstorageOEMs.TheEmulexfamilyofLightPulse™FibreChannelHBAsandOneConnect™ UCNAsprovideITadministratorstheflexibility,performanceandreliabilitytheyneedtokeep pace with demanding and dynamic business environments.

Emulex OneConnect UCNA

The Emulex OneConnect UCNA is a single-chip, high-performance 10GbE adapter with support for TCP/IP, FCoEand iSCSI, enablingoneadapter to supportabroad rangeofnetworkprotocols.OneConnectisdesignedtoaddressthekeychallenges of the evolving data center and improve the overall operational efficiency. OneConnect UCNA is a flexible serverconnectivity platform that enables IT administrators to consolidate multiple

1GbE links onto a single 10GbE link. With support for TCP/IP,FCoE, iSCSIand InternetWideAreaRDMAProtocol(RoCE)onasingleplatform,ITadministratorscan also meet the connectivity requirements of all networking,storageandclusteringapplications.Suchflexibilitysimplifiesserverhardwareconfigurationsandsignificantly reduces standard server configurationsdeployed in the data center.

For greater performance, at adapter and serverlevel, OneConnect leverages iSCSI and FCoE offload technology. This notonly improves adapter performance, but also leaves more of the server’s CPU cycles available for application workload processing. The end result ismore effective utilization of existing IT assets, which helps to reduce capital


expenditures.Infact,Emulex’sOneConnectUCNAdesignissoinnovativethatNetworkComputingnotonly recognized itas the “NewProductof theYear”butalsothe“NetworkInfrastructureProductoftheYear”.Butthetruemeasureof OneConnect’s success has been its acceptance and deployment in datacenters large and small.

Emulex LightPulse Fibre Channel HBAs

EmulexLightPulseHBAsleverageeightgenerationsofadvanced,field-proventechnologiestodeliveradistinctivesetofbenefitsthat are relieduponby theworld’s largest enterprises. Fromtheuniquefirmwareupgradeablearchitecture,tothecommondrivermodel,Emulexisconsideredtoprovidethemostreliableand scalable FibreChannelHBAs, andhas received variousindustry accolade

EmulexLightPulse8Gb/sFibreChannelHBAsprovidethebandwidthrequiredtosupporttheincreaseindatatrafficbroughtaboutbyorganizationsthatare:

- Consolidatingserverresourcesthroughdeploymentofvirtualizationandblade server technologies

- Leveraginghigherperformancenext-generationserverplatforms

- Deploying or enhancing storage networking infrastructure to address transaction intensive and data streaming applications

- Increasingdatacenterpowerefficiency


EmulexFibreChannelHBAsaredesignedwiththeenterprisecustomerinmind.Working in close collaborationwith IT organizations and system-levelOEMs,Emulex integrates features that streamline the deployment and simplify themanagementofFibreChannelHBAswithinthedatacenter.

Interoperability

Emulexserverconnectivitysolutionsarebasedonindustrystandards.Emulexworkscloselywithserver,switch,storageandsoftwareOEMstoensurehighestlevel of interoperability within heterogeneous data center environments. This is justoneof the reasonswhyEmulexHBAsandUCNAshavebeenbroadlyadoptedanddeployedbyITorganizationslargeandsmall.

Broad Operating System Support with Investment Protection

Emulex provides support for the major enterprise class operating systems.Leveraging the exclusive “common driver” model, Emulex ensures FibreChannel driver interoperability between generations of LightPulse HBAs andOneConnectUCNAs.ThisapproachhelpstopreserveITinvestment,aswellassimplifying redeployment.

Emulex’s Service Level Interface (SLI™) architecturewas developed to allowdeployment of new firmware releases on one server or multiple serversthroughout the network without rebooting. Firmware independence and thecommondrivermodelalsomeanthatEmulexadapterscaneasilyberedeployedin servers running different operating systems.

OneCommand™ Manager – Centralized, Multi-protocol Adapter Management

Emulexserverconnectivitysolutionsarenotonlydesignedforperformanceandscalability,butalsomanageability.EmulexconsolidatedthemanagementofitsHBAs and UCNAs under a singlemanagement application – OneCommandManager.WithOneCommandManager,ITadministratorscanremotelymanageEmulex Fibre Channel, iSCSI, FCoE and NIC resources from a centralizedlocation. Furthermore, powerful diagnostic and automation functions withinthis application further help streamline administration functions, thus improving managementefficiency.

Regardlessoftheprotocol,OneCommandManagersimplifiestheadministration,maintenance and monitoring of server connectivity across the entire data center.

Emulex – The Solution of Choice

Withover25yearsofstoragenetworkingexperience,Emulexserverconnectivitysolutionsdelivertheperformance,flexibility,scalabilityandreliabilityorganizationsneed to address the demands of today’s dynamic business environment.

This experience, combined with close development partnerships with the


industry’sleadinghardwareandsoftwareOEMs,hasmadeEmulex’sfamilyofLightPulseFibreChannelHBAsandOneConnectCNAsthesolutionofchoicefortheenterprisedatacenter.EmulexHBAandCNAsolutionshavebeenqualifiedand are used in a broad range of standard and blade server platforms.

Regardless of whether you are using a pure Fibre Channel network, ortransitioning to a converged network environment using 10GbE, Emulex hastheservertonetworkconnectivitytoaddressyourchallengingneeds.FormoreinformationonEmulexsolutions,pleasevisitEmulex.com.


Chapter 10: Conclusion

Converged networking is an emerging technology that will change theway data center managers install and operate equipment, processes and manpower.Convergednetworkingresultsinanoverlapofnetworkandstorageadministrators’ responsibilities. This guide explains networking and storagebasics to help each administrator better understand the changes resulting from convergednetworkingandhowitwillimpacttheirroleinthedatacenter.Figure10providesanexampleofaconvergednetworkenvironment.

Figure 10:Convergednetworkdeployment

Look to Emulex to provide not only adapters to serve a converged networkenvironment, but also to help educate the industry as it evolves.

For more information on converged networking, download the EmulexConvergenomicsGuidefromEmulex.com.

emulex - management mind meld (a. ordoubadian)

Documents