cisco net workers 2006 - san-1501 - introduction to storage area networking

1© 2006 Cisco Systems, Inc. All rights reserved. Cisco PublicSAN-150112686_05_2006_c2

Introduction to Storage Area Networks (SAN)

SAN-1501

2© 2006 Cisco Systems, Inc. All rights reserved. Cisco PublicSAN-150112686_05_2006_c2 © 2005 Cisco Systems, Inc. All rights reserved.

Session Agenda

• Storage BasicsThe What …

• Fibre Channel Basics… and the How.

• Where to go from here

Main Topics for Today


Storage Basics


Storage Basics

• Storage componentsHost bus adapter

Disks, JBODs, RAID, storage arrays

• Storage topologiesDAS, NAS, and SAN

• Storage protocolsSCSI

Storage transport protocols

Fibre Channel (FC) , FCIP, iSCSI


Storage Components


Host Storage Components

• Host bus adapter (HBA)

• Hardware resident on host server

• Connection interface to disk subsystem

• Connection methodCopper

Optical


Data Storage Components

• Fundamental unit for data storage

• Disk drive typesParallel SCSI

Fibre channel

Advanced Technology Attachment ATA) or Integrated Drive Electronics (IDE)

Serial ATA (SATA)

Other

• Tape drives

I/O Devices—Disk Drives


Storage Interconnects

Connecting Hosts to Storage

• Parallel SCSI copper interconnects

• Optical direct connect

• Fibre Channel switch


The SCSI I/O Channel—Starting Point

• SCSI I/O Channel provides half-duplex pipe for SCSI command, data, and status

• SCSI I/O channel can be internal or external to host

• Multiple SCSI I/O channels can exist within host

• A network approach can scale the I/O channel in many areas (length, devices, throughput)

NIC Adapter SCSI Adapter

NIC Driver

TCP/IP Stack

Adapter Driver

SCSI Generic

Block Device

File System

Applications Half-DuplexI/O Channel

SCSI

SCSI

SCSI

SCSIInitiator

SCSITarget


SCSISCSI

I/O C

hann

el

Target 2

ChannelController

Host (Initiator)

Target 1

The Parallel SCSI I/O Channel

• The SCSI Channel is used to transmit SCSI commands, data, and status

• Multiple devices can exist on the channel each playing the role of SCSI initiator or SCSI target

• Most common channel is the basic parallel SCSI bus which can be internal or external to a host

• Parallel SCSI Specifications:Up to 25m bus length

Shared channel bandwidth upto 320MBps

Up to 16 devices per SCSI bus

Half-duplex operation


SCSISCSI

Target

ChannelController

Host (Initiator)

Net

wor

ked

I/O C

hann

el

Initiator

Networking the I/O Channel

• Same SCSI protocol carried over a network transport via serial implementation

• Transport must not jeopardize SCSI payload (security, integrity, latency)

• Two primary transports to choose from today, namely IP and Fibre Channel

• A networked I/O channel allows for multiple improvements:

Distance limitations greatly increased

Dedicated bandwidth (not shared)

High # of addressable devices

Bandwidth increase (including link bundling)

Network


Fibre ChannelFabric

SCSISCSI

Target

EthernetNIC

Host System

Initiator

Fibre Channel I/O Networking

• Very common method for networking SCSI

• Fibre Channel provides high speed transport for SCSI payload

• Fibre Channel overcomes many shortcomings of Parallel I/O including:

Addressing for up to 16 million nodes

Loop (shared) and fabric (switched) transport

Host speeds of 100 to 400 MBps (1-4 Gbps)

Segments of up to 10km (without extenders)

Support for multiple protocols

• Combines best attributes of a channel and a network together


IP I/O Channel Networking

• IP access to open systems iSCSI and Fibre Channel storage

• iSCSI driver is loaded onto hosts on Ethernet network

• Able to consolidate servers via iSCSI onto existing storage arrays

• Able to build Ethernet-based SANs using iSCSI arrays

• Storage can be mapped from iSCSI to Fibre Channel

iSCSI-EnabledHost (Initiator)

iSCSI Appliance

(Target)

StoragePool (Target)

iSCSIGateway

IPNetwork

FCFabric FC HBA

Attached Host

(Initiator)

iSCSI

iSCSIiSCSI

EthernetNIC

SCSISCSI


Storage Subsystems

• Just a bunch of disks (JBOD)

• Storage arrays

• Redundant array of independent disks (RAID)


Simple Storage SystemsI/O Devices—JBODs

ServerWith SCSI

ServerWith Fibre Channel

Arbitrated Loop(FC-AL)

TRGT 0 TRGT 2

TRGT 1 TRGT 3

TRGT 4

SCSI JBOD

FC-AL

TRGT 3TRGT 4TRGT 5

FC-AL JBOD

SCSI BUS

TRGT 0 TRGT 1 TRGT 2

SCSI CABLE

FC CABLE


Large Storage Systems – Bus TypeI/O Devices—Intelligent Storage Arrays

X BUS

Y BUS

Disk DrivesDisk Drives

Host Bus Adapters Host Bus Adapters

Host ChannelConnections

Dual-ProcessorChannel DirectorCache

System Bus

Dual-ProcessorDisk Director

Disks

CDCDCDCDCDCDCDCDCacheCache

DDDDDDDDDDDDDDDD


Large Storage Systems – Switched Type

Host ChannelConnections

Dual-ProcessorChannel Connectors

SwitchedBackplane

Dual-ProcessorDisk Connectors

Disks

Host Bus Adapters Host Bus Adapters

CacheCache

Disk DrivesDisk Drives

DDDDDDDDDDDDDDDD

CDCDCDCDCDCDCDCD

I/O Devices—Intelligent Storage Arrays


Logical Arrays of Storage

• Redundant Array of Independent Disks (RAID)Word coined by researchers at University of California, Berkeley in 1987

A method to inexpensively put together a set of physical hard drives into an array

Provides fault tolerance by mirroring or parity operations

RAID can be performed using hardware or using host based software

RAID


Types of RAID

• RAID 0: StrippingFor performance and size, two or more hard disks are concatenated together to form a larger volume

• RAID 1: MirroringFor reliability and availability

• RAID 3: Error DetectionFor reliability, availability of data using using a dedicated parity drive

• RAID 5: Error CorrectionFor reliability, availability of both data and parity


RAID 0: Striping

• Data is segmented and split across multiple spindles

• I/O benefitsShort reads and writes, easily handles multiple simultaneous reads

Long reads and writes, single operations split and processed in parallel

• Redundancy—none

• Cost—good (no extra hardware)


RAID 1: Mirroring

• Data is duplicated on multiple spindles

• I/O benefitsShort and long reads with shorter latency

Short and long writes slower due to multiple writes

• Redundancy—good

• Cost—need double the amount of disks


RAID 0+1:

• Data is striped and duplicated on multiple spindles

• I/O benefitsShort and long reads with shorter latency

Short and long writes faster as writes are spread across multiple spindles

• Redundancy—good

• Cost—need double the amount of disks


RAID 1+0:

• Data is duplicated and striped on multiple spindles

• I/O benefitsLess downtime than RAID 0+1

• Redundancy—good• Cost—need double

the amount of disks


RAID 5: Error Correcting

• Data protection using ECC (Error Correction Control) parity spread over all drives

• I/O benefitsShort and long reads at normal speeds

Short and long writes slower due to parity calculations

• Redundancy—better than RAID3

• Cost—needs only one extra disk for an entire logical disk


Storage Topologies


IP Front-End Network SAN Back-End Network

SANIPNetwork

Storage Topologies

• Host to Host• Application to file system• Client to Server• NFS, SMB, CIFS, NCP• NAS, WAFS

• Host to Storage• File system to Device• Program to Device• SCSI, IDE, NTFS, FAT• SAN


Direct Attached Storage (DAS)

• Storage is captive ‘behind’ the server, limited mobility

• Limited scalability due to limited devices

• No efficient storage sharing possible

• Costly to scale; complex to manage

FC

Clients

Direct Attached Storage

Application Servers

Win2k Linux Unix Unix

Tape

FC

LinuxWin2k

SCSI

LAN


Network Attached Storage (NAS)

• Storage is accessed at a file level via NFS or CIFS

• Storage is accessed over an IP network

• Storage devices can be shared between servers

• Files can be shared between users

• TCP can be tuned to optimize for file transport Application

ServersNAS Appliances

orNAS Head Ends

Generic Generic

NAS = Optimized for File I/O

Win2k Linux Unix

LAN


Storage Area Network (SAN)

• Storage is accessed at a block-level via SCSI

• High performance interconnect providing high I/O throughput

• Lower TCO relative to direct attached storage, storage can be shared

• Limited vendor interoperability

• Complex management

Separation of Storage from the Server

Servers

BlockStorageDevices

Fibre Channel

SAN

Clients

LAN


iSCSI Storage Network

• Storage is accessed at a block-level via iSCSI

• Good performance via standard Ethernet NIC

• Enhanced performance with TCP Offload Engine (TOE)

• Lower TCO relative to direct Fibre Channel HBA/Fabric

• Standards based

• Complex management

iSCSI-EnabledHosts (Initiators)

iSCSI Appliance

(Target)

StoragePool (Target)

iSCSIGateway

IPNetwork

FCFabric

FC HBA Attached

Host (Initiator)

iSCSIiSCSI

iSCSI

iSCSI


DAS, SAN, iSCSI, NAS Comparison

DAS

SCSI

Computer System

SCSI Bus Adapter

SCSI Device Driver

Volume Manager

File System

Application

iSCSI Appliance

IP

File System

Application

SCSI Device DriveriSCSI DriverTCP/IP stack

NIC

Volume Manager

NICTCP/IP StackiSCSI LayerBus Adapter

iSCSI Gateway

IP

FC

File System

Application

SCSI Device DriveriSCSI DriverTCP/IP stack

NIC

Volume Manager

NICTCP/IP StackiSCSI Layer

FC HBA

NASAppliance

IP

NICTCP/IP Stack

I/O Redirector

File System

Application

NFS/CIFS

NICTCP/IP StackFile System

Device Driver

Block I/O

NASGateway

IP

NICTCP/IP Stack

I/O Redirector

File System

Application

NFS/CIFS

FC

NICTCP/IP StackFile System

FC HBA

Host/Server

StorageTransport

StorageMedia

SAN

SAN

FC

FC HBASCSI Device Driver

Volume Manager

File System

Application

Computer System Computer System Computer System Computer System Computer System

Block I/O File I/O


Wide Area File Services (WAFS)

IntelligentMessage Suppression

Protocol proxy to handle non-critical messages

Operation batching and message bundling

Decouple the user experience from the WAN

Data IntegrityMaintain data integrity at all costs

Never compromise security of the data

Guarantee support for all protocol semantics

Data Distributionand Caching

Validate and serve information locally

Prepopulate via preposition and on-demand

Asynchronous writes to mask large transfers

Link and ThroughputOptimization

Compress all messages

Pipeline multiple segments into a common window

Multiplex over many sockets if necessary

WAFS Provides File Services Between a Remote Client and a Centralized Server


Storage Protocols


Small Computer System Interface—SCSI

• SCSI is a standard that defines an interface between an initiator (usually a computer) and a target (usually a storage device such as a hard disk)

• Interface refers to connectors, cables, electrical signals, optical signals and the command protocol that allow initiators and targets to communicate

• Logical Unit Number (LUN): A 64-bit field within SCSI that identifies the Logically addressable Unit within a target SCSI device


SCSI Command Protocol

• SCSI command protocol is the de facto standard that is used extensively in high-performance storage applications

• The command part of SCSI can be: Transported over a Fibre Channel storage area network

Encapsulated in IP and carried across IP networks

• To understand the finer points involved with transporting SCSI across a network with FC or Ethernet, the basics of SCSI must be well understood


Standards

• SCSI has evolved since it was introduced as SASI in 1979 by Shugart Associates – it was approved as a standard by ANSI in 1986 and is now referred to as SCSI-1

• SCSI-2 was approved by X3 in 1990 and by ANSI in 1994

• SCSI-3 refers to a collection of standards, each of which defines a very specific part of SCSI: physical interface, transport interface, command interface, architecture model, programming interface, etc.

Also known as SCSI Architecture Model – 3 (SAM-3)


SCSI Architecture Model

SAN


SCSI I/O Channel

The SCSI I/O Transaction

• The SCSI protocol forms the basis of an I/O transaction

• The channel provides connectivity between communicating devices in a SCSI transaction

The following shows two sample SCSI exchanges:

Host (Initiator) Disk (Target)

READDATA DATA DATA

WRITEDATA DATA DATA

STATUS

STATUS

SCSI I/O ChannelSTATUSSTATUS

Disk (Target)Host (Initiator)

SCSI READ OPERATION

SCSI WRITE OPERATION


SCSI Commands

• Data transferREAD, WRITE

• Commands used in boot/discovery:REPORT LUNS

INQUIRY

TEST UNIT READY


SCSI Read

1. Send SCSI Cmd issued by initiator – the command sent is READ;

2. SCSI Command Received by target;

Data transfers occur during the ‘working’ phase between initiator and target;

3. Send Command Complete is returned by the target;

4. Command Complete Receivedby target.

…


SCSI Write

1. Send SCSI Cmd issued by initiator—the command sent is WRITE;

2. SCSI command receivedby target;

3. Target returns TRGT-RDY;4. Initiator receives TRGT-RDY;

Data transfers occur during the ‘working’ phase between initiator and target;

5. Status complete is returned by the target;

6. Status complete received by initiator


Storage Transport Protocols

• Protocols used to transport SCSI Command and DataParallel SCSI

Fibre Channel

FCIP

iSCSI


Storage ProtocolsProtocol and Transport Stack

Parallel SCSIInterfaces Fibre Channel Ethernet

IP

TCP

Parallel SCSIInterfaces Fibre Channel iSCSI

SCSI Commands, Data, and Status

SCSI Block Commands

SCSI StreamCommands

Other SCSI Commands

SCSI Applications (File Systems, Databases)

FCIPFibre Channel


SCSISCSI

I/O C

hann

el

Target 2

ChannelController

Host (Initiator)

Target 1

The Parallel SCSI I/O Channel

• The SCSI Channel is used to transmit SCSI commands, data, and status

• Multiple devices can exist on the channel each playing the role of SCSI initiator or SCSI target

• Most common channel is the basic parallel SCSI bus which can be internal or external to a host

• Parallel SCSI Specifications:Up to 25m bus length

Shared channel bandwidth upto 320MBps

Up to 16 devices per SCSI bus

Half-duplex operation


Storage Protocols


IP

TCP



SCSI Block Commands

SCSI StreamCommands

Other SCSI Commands


FCIPFibre Channel

Fibre Channel Protocol and Transport Stack


What Is It? Fibre Channel

Channels• Connection service• Physical circuits• Reliable transfers• High speed• Low latency• Short distance• Hardware intense

Networks• Connectionless• Logical circuits• Unreliable transfers• High connectivity• Higher latency• Longer distance• Software intense

Fibre Channel• Circuit and packet

switched

• Reliable transfers

• High data integrity

• High data rates

• Low latency

• High connectivity

• Long distance


FICON

FC SingleByte Command

Sets

IP

Link Encapsulation

(FC-LE)

ULP (Upper Level Protocol)(sample – there are more) SCSI-3

SCSI-3 Mapping(SCSI-FCP)

FC-4 SNMP Mapping(FC-SNMP)

FC-3 Common Services

FC-0

FC-1

FC-2 Fibre Channel Physical & Signaling

Interface(FC-PH, FC-PH2,

FC-PH3)Physical Interface

Encode / Decode

Framing Protocol FC-AL FC-AL-2 FC-SW-2

Copper Optical

8B/10B Encoding

Fibre Channel Protocol Architecture


Fibre Channel Functions

• FC-0• Defines the physical interface characteristics

Signaling rates, cables, connectors, distance capabilities, etc.• FC-1• Defines how characters are encoded/decoded for transmission

Transmission characters are given desirable characters• FC-2• Defines how information is transported

Frames, sequences, exchanges, login sessions• FC-3 • Place holder for future functions• FC-4• Defines how different protocols are mapped to use Fibre Channel

SCSI, IP, Virtual Interface architecture, FICON, others

Structure Is Divided into 5 Levels of Functionality


OX_ID & RX_ID

FrameFields

SEQ_ID

SEQ_CNT

ULP Information Unit

Exchange

Sequence Sequence Sequence

Frame Frame Frame

Fibre Channel FC-2 Hierarchy

• Multiple exchanges are initiated between initiators (hosts) and targets (disks)

• Each exchange consists of one or more bi-directional sequences

• Each sequence consists of one or more frames• For the SCSI3 ULP, each exchange maps to a SCSI command


Idles* SOF Frame Header Data Field CRC EOF Idles*

General FC-2 Frame Format

Frame Content

0-528 Transmission Word

(4)(4)(0-2112)(24)(4)

Fibre Channel Frame Format

• All FC-2 frames follow the general frame format as shown below

• Idles are ‘Ordered Sets’ used for synchronization and basic signaling

• SOF – Start-of-Frame, EOF – End-of-Frame

* 6 Idle words (24 bytes) requires by TX2 Idle words (8 bytes) guaranteed to RX


Fibre Channel Storage Area Network (SAN)

Switch ASwitch A

FC

EFFC

FCF

Switch BSwitch B

FC

FEFC

FCF

= Fibre Channel= F_Port

FCFHost

Storage

Storage

Host


Storage Protocols

FCIP Protocol and Transport Stack


IP

TCP



SCSI Block Commands

SCSI StreamCommands

Other SCSI Commands


FCIPFibre Channel


IP

Storage Protocols - FCIP

• Primary use is for Storage-to-Storage connectivity via an IP WAN/MAN

• The FCIP Link carries encapsulated fibre channel traffic between Link End Points over an IP network by using TCP on port 3225

• The result is a virtual Inter Switch Link (ISL) between FC Fabrics

• FC frame not changed

TCPFCIP

FCSCSI

Data

FCIP Frame Format


FCIP SAN Interconnect

Switch ASwitch A

FC

EFFC

FCF

Switch BSwitch B

FC

FEFC

GEVE

Switch CSwitch C

FC

EVEGE

GEF

= Fibre Channel= F_Port= E_Port= Virtual E_Port= Gigabit Ethernet

FCFEVEGE

IP

Virtual ISL

FCIP Link

StorageStorageHost

Host


Storage Protocols

iSCSI Protocol and Transport Stack


IP

TCP



SCSI Block Commands

SCSI StreamCommands

Other SCSI Commands


FCIPFibre Channel


Storage Protocols

• Primary use is for Host-to-Storage connectivity via an IP LAN

• SCSI command and data are encapsulated into iSCSI by adding a special header

• iSCSI data is encapsulated into a TCP packet

• IP is the transport protocol

Storage Protocols—iSCSI

IPTCP

iSCSISCSI

Data


iSCSI Architecture

• SCSI deviceIP host

iSCSI node initiator

FC storage

iSCSI node target

• SCSI portiSCSI initiator port

iSCSI target port

• Network portalAny network interface with TCP/IP

Provides physical IP network access`

iSCSI Target Port

Network PortalIP AddressTCP Port

iSCSI Initiator Port

Logical Units

iSCSI Target Port



SCSI Device (iSCSI Node – Initiator)

iSCSI—SCSI Relationship

SCSI Device (iSCSI Node – Target)

Network PortalHBA/NIC

Network PortalHBA/NIC

IP Network


iSCSI Host Attachment

Switch ASwitch A

FC

EFFC

FCF

Switch BSwitch B

FC

FEFC

GEVT

= Fibre Channel= F_Port= E_Port= Virtual Target= Gigabit Ethernet

FCFEVTGE

IP

iSCSI

TCP/IP Link

StorageHost iSCSI

Host


Storage Protocols Summary

• Protocols used to transport SCSIParallel SCSI – Legacy host to storage bus topology

Fibre Channel – Robust serial transport network for SCSI

Primary transport for Storage Area Networks

FCIP – FC-2 encapsulation in TCP/IP

SAN-to-SAN Connectivity

Typically used in the WAN/MAN

iSCSI – TCP/IP transport for SCSI protocol

Host-to-SAN Connectivity

Typically used in the LAN

Storage Transport Protocols


Fibre Channel Basics


Fibre Channel Basics

• Fibre Channel Physical AttributesPortsLinksNodesFabrics

• Fibre Channel Operational CharacteristicsNamingAddressingFabric ServicesRoutingZoning


FC Physical Attributes


Physical

• The main physical objects in Fibre Channel are:Ports

Links

Nodes

Fabric


Fabric Switch

FL_Port

G_Port

G_Port

G_Port

E_Port

F_Port

F_Port

F_Port

E_Port

NodeN_Port

NodeN_Port

NodeN_Port

Node NL_Port

Node NL_Port

Node NL_Port

Port Type Examples


Fibre Channel Port Types

• ‘N’ port: Node ports used to connect devices to switched fabric or point to point configurations.

• ‘F’ port: Fabric ports residing on switches connecting ‘N’ port devices

• ‘L’ port: Loop ports are used in arbitrated loop configurations to build networks without FC switches. These ports often also have ‘N’ port capabilities and are called ‘NL’ ports.

• ‘E’ port: Expansion ports are essentially trunk ports used to connect two Fibre Channel switches

• ‘GL’ port: A generic port capable of operating as either an ‘E’ or ‘F’ port. Its also capable of acting in an ‘L’ port capacity. Auto Discovery.

N N

N F

NL FL

L L

E E


Inter-Switch Link (ISL)

• The interconnection between switches is called the Inter-Switch Link (ISL)

E_Port to E_Port

• Supports all classes of serviceClass 1, 2, 3, and a special Class F

• FC-PH permits consecutive frames of a sequence to be routed over different ISL links for maximum throughput

E E


Other standard and Cisco Specific Port Types

• TE Trunking E-Port that to carry Virtual SAN’s

• Fx Ports found on Over-subscription Port

• SD Span Destination port for Analyzer

• ST Span Tunnel port used in Remote Spanfunction

• TF Trunking F-Port

• TL Translation Loop (Private to Public addressTranslation)


N_Port

Device

N_Port

Host/ Device Interface

Serial Data Out Serial Data In

Host

FC

Can be more then one N_Port on the device


Fibre Channel Link

• A link consists of2 unidirectional “fibers” transmitting in opposite directions

May be either:

Optical fiber or Copper

• Transmitters may be:Long wave laser

Short wave laser

LED

Electrical


Node

• The equipment which contains one or more• N_Port or NL_Port (topology dependent)• May be:

Computer (HBA)Controller (Port on Disk Subsystem)Device (SCSI FC to Parallel converter)

• Is NOT a switch fabric device

N_Port

Link

N_Port

Link

N_Port

Link

N_Port

Link

Controller


FC Communications Model

• Point to point

• N_Port to N_Port

• Flow controlled

• Acknowledged

TX

RX

N_Port

Node Node

Link

N_Port

Host

FC

Host

FCRX

TX

Node Node


Fibre Channel Fabric

• The entity which interconnects N_Ports

• Provides routing based on destination address

• Fabric may be:Point to point: No routing required

Arbitrated loop: Routing is distributed throughout attached L_Ports

Switched: Routing provided by switch

Fabric


Point to Point

Communications Model• Source to destination

• Host Bus Adaptor (HBA) keeps in memory a Port Login Table of each N_Port known to it on the network

Node Node

Transmitter

Receiver

Receiver

Node Node

N_Port

Transmitter

N_Port


Communications Model—Source to Destination Based on Address Routing Distributed in the NL_Ports on the Loop

Arbitrated Loop

NL_Node “A”

NL_Node “B”

Link

A B Still Pt. to Pt. Communication


Switched Fabric

N_Port N_PortN_Port N_Port N_PortN_Port

Fabric

Switch

Communications Model—Source to Destination Based on Address Routing through the Fabric. Still a Pt. to Pt. connection. FSPF routing required when more then 2 switches make up the Fabric

A B

A B


FC Operational Characteristics


Worldwide Names

• Each switch element is assigned a WWN at time of manufacture

• Each switch port is assigned a WWN at the time of manufacture

• During FLOGI the switch identifies the WWN in the service parameters of the accept frame and assigns a Fibre Channel ID (FCID)

• These address assignments can then correlate each fabric port with the switch element


Fabric Addressing

• The 24 bit FCID address is partitioned into 3 fieldsDevice

Area

Domain

• This partitioning helps speed up routing

• Switch element assigns the address to N_Ports

• Address portioning is transparent to N_Ports

DeviceAreaSwitch Domain

8 bits 8 bits 8 bits

Switch Topology


Directory Server

• Repository of information regarding the components that make up the Fibre Channel network

• Located at address ‘FF FF FC’ (Some readings call this the name server)

• Components can register their characteristics with the directory server

• An N_Port can query the directory server for specific information

Query can be the address identifier, WWN and volume names for all SCSI targets


Fabric Controller

• Each switch has a fabric controller

• Assigned address ‘FF FF FD’Every fabric controller in the fabric has the same address

It is the N_Port within the switch

Responsible for managing fabric, initialization, routing, setup and teardown of Class-1 connections

• Responsible to receive request and generate responses for the switch fabric

Information must be consistent independent of which fabric controller responds to a request


Extended Link Services

• Extended link services provide a set of protocol functions used by the port to specify a function or service at another port

Usually sent from N_Port to F_port to perform needed request

The R_CTL field of the first word will be set to 0x22 to indicate an extend link service request

Many ELS services will return a payload in response some have no reply


Extended Link Services

Some of the more important and most used ELS commands are:

• FLOGI F_Port Login

• PLOGI N_Port Login

• FAN Fabric Address Notification

• PRLI Process Login

• PRLO Process Logout

• SCN State Change Notification

• SCR State Change Registration

• RSCN Registered State Change Notification


Fabric Configuration: PS Selection

• A principal switch shall be selected whenever at least one inter-switch link (A link between two E_Port) is established

• The selection process chooses a principal switch, which is then designated to assign domain identifier to all the switches in the fabric, and any who join later the fabric later on

• The principal switch selection can be triggered by anyone of the following events

Switch boot and Exchange Fabric Parameters (EFP)

Build Fabric (BF)

Reconfigure Fabric (RCF)


Fabric Configuration Disruptive/Non-Disruptive

• One of the following three conditions can trigger BF (non-disruptive) or RCF (disruptive)

Two disjoint fabrics are combined together

A principal ISL fails (upstream or downstream)

A switch with Domain_ID request for another Domain_ID

• Whenever a switch receives a BF/RCF, the switch starts F_S_TOV timer and enters the BF/RCF state; It forwards BF/RCF out of all E_ports except the incoming port (only once) and wait for the timer to expire

• When the timer expires, BF/RCF propagation state is left and principal switch selection begins

• BF is not a disruptive process

• RCF is a disruptive process


Fabric Routing: FSPF

• For FSPF a domain ID identifies a single switchThis limits the max number of switches that can support in the Fabric to 239 when FSPF is supported

Treat each Cisco MDS VSAN as separate Fabric each potentially supporting 239 switches

• FSPF performs hop-by-hop routingEach Cisco MDS VSAN runs it’s only FSPF process

Routing between VSAN’s is done with Inter VSAN Routing (IVR)

• FSPF supports hierarchical path selectionProvides the scalable routing tables in large topologies


Fabric Routing: FSPF

• Everyone says HELLO to their neighbor, on all initialized ISLs

• The neighbors say HELLO back, unless they are dead

• When the HELLO packet is received with both originator and recipient domain id, the two way communication is done and:

The ISL is active

The ISL may be available as a two-way path for frames


FSPF Characteristics

• Uses FSPF as the routing algorithm

• FSPF routes traffic based on destination domain ID

• FSPF uses total cost as the metric to determine most efficient path

• Static routes can be applied


Fabric Switch

FL_Port E_Port E_Port

NodeN_Port

NodeN_Port

NodeN_Port

Node NL_Port

Node NL_Port

Node NL_Port

F_Port

Zone_A

F_Port

F_Port

Zone

_B

Fibre Channel Fabric Zoning

• Fibre Channel zoning is used to restrict access within the fabric to certain nodes/resources

• Zones may overlap devices


Fibre Channel Fabric Zoning

• Zoning operationZone members “see” only other members of the zone

Zones can be configured dynamically based on WWN

Devices can be members of more than one zone

FC-AL zoning allows the creation of private loops on a single hub

Switched fabric zoning can take place at the port or device level

Based on physical switch port

Based on device WWN

Based on LUN ID

• BenefitsSecured device access

Allows operating system co-existence

SAN

Disk1

Host2Disk4

Host1

Disk2 Disk3

ZoneA

ZoneB

ZoneC


Fibre Channel Basics Summary

• Fibre Channel is a very robust, hierarchical standard

• Fibre Channel utilizes a Point-to-Point communications model irrespective of the topology

• Fibre Channel includes a full set of services for naming, addressing, building, and managing fabrics

• Fibre Channel utilizes FSPF, an OSPF like routing protocol to route traffic

• Fibre Channel Zoning is a method of logically grouping devices within a given fabric

We have only scratched the surface of FC Protocol here!


√ SAN-2501: Fibre Channel Storage Area Network Design√ SAN-2602: SAN Migration and Interoperability√ SAN-2604: SAN Extension Design and Operation√ SAN-2605: iSCSI SAN Design and Operation √ SAN-2606: SAN Virtualization √ SAN-3607: Advanced SAN Design, Virtual Fabric and Fabric Routing√ SAN-3608: Advanced SAN Troubleshooting √ LABSAN-2251, 2252, 2253, 2254, and 2265: Cisco MDS 9000 Labs√ TECSAN-2601 SAN Fundamentals, Protocols, and Architecture

Where Do I Go from here ?


Q and A


Recommended Reading

• Continue your Cisco Networkers learning experience with further reading from Cisco Press

• Check the Recommended Reading flyer for suggested books

Available Onsite at the Cisco Company Store


Complete Your Online Session Evaluation

• Win fabulous prizes; Give us your feedback

• Receive ten Passport Points for each session evaluation you complete

• Go to the Internet stations located throughout the Convention Center to complete your session evaluation

• Drawings will be held in theWorld of Solutions

Tuesday, June 20 at 12:15 p.m.

Wednesday, June 21 at 12:15 p.m.

Thursday, June 22 at 12:15 p.m. and 2:00 p.m.

cisco net workers 2006 - san-1501 - introduction to storage area networking

Documents