brocade storage
TRANSCRIPT
-
8/15/2019 Brocade Storage
1/44
53-1003534-01
October 2014
rocade VCS Storage
Deployment Guide
-
8/15/2019 Brocade Storage
2/44
2014, Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, the B-wing symbol, Brocade Assurance, ADX, AnyIO, DCX, Fabric OS, FastIron, HyperEdge, ICX, MLX, MyBrocade, NetIron,
OpenScript, VCS, VDX, and Vyatta are registered trademarks, and The Effortless Network and the On-Demand Data Center are trademarks
of Brocade Communications Systems, Inc., in the United States and in other countries. Other brands and product names mentioned may be
trademarks of others.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning anyequipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document
at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be
currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in
this document may require an export license from the United States government.
The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the
accuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that
accompany it.
The product described by this document may contain open source software covered by the GNU General Public License or other open
source license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable to
the open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.
http://www.brocade.com/support/oscd
-
8/15/2019 Brocade Storage
3/44
Contents
Brocade VCS Storage Deployment Guide...................................................................................5
Preface..............................................................................................................5
Storage Technology Overview.......................................................................... 5
Direct-Attached Storage........................................................................6
Network-attached storage.....................................................................6
Storage Area Network...........................................................................7
Deployment options........................................................................................ 13
Rack server and ToR connectivity to the FC SAN using FC routing...14
Rack server and ToR connectivity to an FC SAN using an access
gateway......................................................................................... 33
FC/FCoE storage connectivity from ToR, rack server, and ToR
connectivity to storage...................................................................37
Brocade VCS Storage Deployment Guide 3
53-1003534-01
-
8/15/2019 Brocade Storage
4/44
4 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
5/44
Brocade VCS Storage Deployment Guide
● Preface..............................................................................................................................5
● Storage Technology Overview..........................................................................................5
● Deployment options........................................................................................................ 13
Preface
This document is a deployment guide for integrating a storage area network (SAN) into a Brocade VCS
fabric. It is written for technology decision-makers, architects, systems engineers, NOC engineers and
other experts responsible for network upgrades and migration.
This document explains the purpose and operation of a SAN, provides the configuration procedures,and presents three common use cases for deploying a SAN in a VCS fabric.
This document provides step-by-step examples to prepare, perform, and verify the deployment of
storage in a VCS fabric. It is assumed that the reader is familiar with establishing console access and
entering commands using the Brocade CLI. For information about the Brocade CLI, refer to the Network
OS Administration Guide.
Document History
Date Version Description
October 2014 1.0 Initial version
Storage Technology Overview
Data storage on hard disks, tape drives, and so forth, evolved quickly after the introduction of personal
computers. The Small Computer System Interface (SCSI) protocol was developed to provide a
standardized method for connecting the CPUs in PCs and servers to disk storage. Over time, three
methods of connecting storage devices and arrays have been developed to CPU hardware.
• Direct-Attached Storage (DAS)
• Network Attached Storage (NAS)
• Storage Area Network (SAN)
Brocade VCS Storage Deployment Guide 5
53-1003534-01
-
8/15/2019 Brocade Storage
6/44
Direct-Attached Storage
FIGURE 1 Direct-Attached Storage
Figure 1 illustrates DAS, where the CPU is attached directly to the disk drives over the system bus,
using the SCSI protocol to write and read blocks of data to and from disk storage devices
Network-attached storage
FIGURE 2 Network-attached storage
Direct-Attached Storage
6 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
7/44
Figure 2 illustrates NAS, where servers and other storage consumers are connected to a storage array
through an Ethernet LAN. Servers communicate with the storage arrays using the following protocols:
• Network File System (NFS): NFS reads and write files to a storage device, which has a controller
that converts the file to blocks of data (block I/O) that are written and read to the local disks on the
device.
• Internet Small computer system interface (iSCSI): iSCSI sends and receives blocks of data from
and to the disk on the storage device over an IP network.
SCSI reads and writes data in blocks directly from and to the storage device. Block I/O is faster
because it directly accesses the storage device.
Storage Area Network
FIGURE 3 Storage Area Network with LAN
Figure 3 illustrates a SAN, which provides access to consolidated, block level data storage over a FC
(FC) network. A SAN typically has its own network of storage devices that are generally not accessible
through the LAN by other devices. The SAN network typically has FC switches to interconnect thestorage devices and servers.
Storage Area Network
Brocade VCS Storage Deployment Guide 7
53-1003534-01
-
8/15/2019 Brocade Storage
8/44
FIGURE 4 Data center with LAN and SAN
Figure 4 shows a data-center where servers have both LAN and SAN connectivity. Servers are
connected to the storage arrays on the FC network using host bus adapters (HBAs). On the LAN,
servers use Ethernet ports on a network interface card (NIC) to connect to applications and clients.
Converged Networks
FC over Ethernet (FCoE) allows the transmission of FC traffic over an Ethernet network so servers do
not need separate FC HBAs to connect to the FC network. Servers in a converged network use
converged network adapters (CNAs) which provide both Ethernet and FC ports. FcoE allows a
converged network to carry both IP and FC traffic, combining the functionality of a LAN and a SAN.
Many modern data-centers use converged LAN and SAN networks working together as shown in
Figure 5 .
Converged Networks
8 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
9/44
FIGURE 5 Converged networks
Fibre Channel
Fibre Channel (FC) is the technology used to build SANS. FC standards are defined by T11 of the
International Committee for Information Technology Standards.
FC was developed to address the limitations of traditional Ethernet networks, which in the past hadlower speed, and had lower reliability because it is based on collision detection which causes some loss
of frames even under normal operation. FC was developed to be lossless and to provide higher speed
and lower latency, starting with a speed of 1 Gb/s, which has been increased over time to 2, 4, 8, and
16 Gb/s.
FC Protocol (FCP) is a transport protocol similar to TCP that transports SCSI commands over FC
networks.
FCoE
FC over Ethernet (FCoE) is an encapsulation protocol that enables the transport of FC storage traffic
over a version of Ethernet called Data Center Bridging (DCB). FCoE encapsulates the entire FC frame,
keeping it intact and unchanged, into DCB frames.
DCB is an enhanced Ethernet version that enables the convergence of various applications in data
centers (LAN, SAN) into a single interconnected network.
FCoE provides a method of encapsulating FC traffic over a physical Ethernet link. FCoE frames use an
EtherType of 0x8906 and FCoE Initialization Protocol (FIP) uses an Ethertype of 0x8914. FIP enables
FCoE SAN traffic and legacy LAN Ethernet traffic to be carried on the same link.
FC frames are encapsulated in an Ethernet frame and sent from one FCoE-aware device across an
Ethernet network to a second FCoE-aware device. The FCoE-aware devices may be FCoE end nodes
(E-nodes) such as servers, storage arrays, or tape drives on one end and FCoE Forwarders on the
Fibre Channel
Brocade VCS Storage Deployment Guide 9
53-1003534-01
-
8/15/2019 Brocade Storage
10/44
other end. FCoE Forwarders are switches that provide SAN fabric services and may also provide
FCoE-to-FC bridging.
FC Ports
Figure 6 illustrates the FC port types that are used in storage networks.
FIGURE 6 FC Port Types
The following summarizes the function of each FC port types:
• N_port is a port on the node (such as a host or storage device); also known as a node port.
• F_port is a port on the switch that connects to connects to an N_port on a node; also known as
fabric port
• E_port is the connection between two FC switches; also known as an expansion port. When
E_ports between two switches form a link, that link is referred to as an inter-switch link (ISL)
• EX_port provides the connection between a FC router and a FC switch. On the side of the switch it
looks like a normal E_port, but on the side of the router it is an EX_port.
When connecting FC ports between two fabrics, FC traffic needs to be routed. In Figure 6 , FC ports
on VCS Fabric are connected to FC ports on the Fabric OS (FOS) Fabric. This requires FC routing on
the FOS fabric that does the FC routing.
On an VCS fabric, FC ports come up as E_ports and on an FC router, FC ports come up as EX_ports.
Worldwide names
A worldwide name (WWN) is a unique identifier used in FC storage networks. The WWN is similar to a
MAC address in an Ethernet LAN except that it is 8 bytes in length. There are two types of WWNs,
worldwide node names (WWNs) and worldwide port names (WWPNs). The WWNN is used to identify
the switch or the host, while the WWPN is used to identify individual ports on the switch or the host.
FC Ports
10 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
11/44
FC network address
The FC network address, called the FCID, is a 3 byte address. When a node is attached to the fabric, a
unique 24-bit address is assigned to the node. The FCID, as illustrated in Figure 7 , contains Domain
ID, Area ID and Node Address fields. The FCID is used to route traffic through the fabric.
FIGURE 7 FCID
Domain ID
In a VCS fabric, rbridge IDs are used to discover fabric creation problems, such as duplicate IDs. The
rbridge ID of a cluster unit is equal to the Domain ID of a FC switch.
Rbridge ID assignment uses the Domain ID assignment protocols used in the FC SAN. The Request for
Domain ID (RDI) and Domain ID Assignment (DIA) protocols ensure that a single switch, the principal
switch, centrally allocates the Domain IDs for every rbridge in the fabric and detects any Domain ID
conflicts. In case of a conflict, the node with a duplicate Domain ID is segmented from the fabric and the
user must resolve the conflict.
Device login
A FC device can be a storage server, host, or switch. When new devices are introduced into the fabric,
powered on, and connected, the following logins occur:
1. FLOGI/FCoE login: Fabric Login (FLOGI) establishes a 24-bit FCID for the device logging in, and
establishes buffer-to-buffer credits and the class of service supported.
2. PLOGI: Port Login (PLOGI) logs the device into the name server to register and query for devices
that share the same zone. During the PLOGI process, information is exchanged between the new
device and the fabric. The following types of information exchanges occur:
• SCR: State Change Registration (SCR) registers the device. If a change in the fabric occurs, such
as a zoning change or a change in the state of a device to which this device has access, the
device receives a Registered State message.
• Change Notification Registration (RSCN): The device exchanges registration information with the
name server.
• Query: The devices queries the name server for information about devices it can access.
Zoning
Zoning is a fabric-based partition that creates a logical group of devices. Zoning provides security
because devices outside the zone cannot access any device inside the zone and vice versa. Zoning
also reduces SCR broadcasts from devices because it restricts all the broadcast traffic to a specific
zone.
You can use zoning to partition the FCS fabric for a number of reasons, including the following:
FC network address
Brocade VCS Storage Deployment Guide 11
53-1003534-01
-
8/15/2019 Brocade Storage
12/44
• Partition the network into zones, such as winzone and unixzone, to keep Windows servers and
storage devices separated from UNIX servers and storage.
• Logically consolidate equipment for efficiency or to facilitate time-sensitive functions. For example,
you can create a temporary zone to back up non-member devices.
A device in a zone can communicate only with other devices connected to the fabric within the same
zone. A device not included in the zone is not available to zone members. When zoning is enabled,
devices that are not included in any zone configuration are inaccessible to all other devices in the
fabric.
Zones can be configured dynamically and can vary in size, depending on the number of fabric
connected devices, and devices can belong to more than one zone.
FIGURE 8 FC Zones
Logical S N zones
A Logical SAN (LSAN) consists of zones in two or more edge or backbone fabrics that contain the
same devices. LSANs essentially provide selective device connectivity between fabrics without
merging the fabrics.
Logical SAN zones
12 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
13/44
FIGURE 9 Logical SAN zoning with FC router
FC routers provide mechanisms to manage inter-fabric device connectivity through extensions to
existing switch management interfaces.
Deployment options
The use cases that can be used for deploying zones in a VCS fabric, and which are described in this
document, include the following:
• Rack server and Top of Rack (ToR) connectivity to the FC SAN using FC routing from the ToR
• Rack server and ToR connectivity to the FC SAN using an access gateway
• FC/FCoE storage connectivity from ToR server and ToR connectivity to storage
Deployment options
Brocade VCS Storage Deployment Guide 13
53-1003534-01
-
8/15/2019 Brocade Storage
14/44
Rack server and ToR connectivity to the FC SAN using FC routing
FIGURE 10 FC routing from the ToR
After a rack server connects to a storage device in the SAN, a Logical Unit Number (LUN) on the
storage device can be mapped to the server. This allows the server to store and read data on the
remote LUN as required. The configuration required includes the following:
• Configure the server with redundant links to redundant ToR switches in the VCS fabric
• Configure the VCS fabric to access the storage network
• Configure a LUN on the storage device
• Connect FC redundant links to storage switches
You can also configure zoning to provide access control for specific servers or LUNs.
The following summarizes the steps to configure a LUN on the storage device and map it to a server:
• Configure the server network
• Configure the storage network
• Configure a LUN on the storage device
• Map a LUN on the server
Configuring the server network
Server network is nothing but bringing up the VCS fabric using brocade 6730s and 6720s/8770s.
Following are the steps involved in bringing up server network.
Step 1: Configure the VCS fabric.
Step 2: Configure FCoE on the edge interfaces.
Step 3: Configure FC connectivity to the storage network.
Step 4: Configure the LSAN_Zone.
Step 5: Configure FCoE connectivity.
Step 6: Verify the FC interfaces and ISLs.
Step 7: Verify that the ISDL ports come up as E_ports.
Step 8: Configure the LSAN zoning.
Rack server and ToR connectivity to the FC SAN using FC routing
14 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
15/44
Procedure
Step 1: Configure the VCS fabric.
A VCS Fabric combines Ethernet, FCoE and FC technology to create virtualized, flexible, and
converged data center networks. When two or more Brocade VDX fabric switches are connected, they
form an Ethernet fabric and exchange information among each other using distributed intelligence. Tothe rest of the network, the Ethernet fabric appears as a single logical chassis.
VCS fabric formation has the following characteristics:
• All switches in the VCS fabric have the same VCS ID.
• Each switch has a unique rbridge ID in the range 1 to 239.
• The VCS fabric uses a link state routing protocol, First Shortest Path First (FSPF) to create the
network topology.
• A broadcast tree is established to distribute fabric broadcast and multicast traffic.
Before connecting the VDX switches, configure the following:
• Set the VCS Fabric ID and enable VCS Fabric mode.
• Set the rbridge ID.
The following is the configuration to bring up the VCS fabric on TOR3 and TOR4 switches. A similar
configuration should be applied on the EOR1, EOR2, TOR5 and TOR6 devices. Rbridge IDs must be
unique; in this example they are configured as follows:
• EOR1 Rbridge ID 1
• EOR2 Rbridge ID 2
• TOR1 Rbridge ID 3
• TOR2 Rbridge ID 4
TOR1TOR2TOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Do youwant to continue? [y/n]:y….TOR1#TOR1# conf tEntering configuration mode terminalTOR1(config)# rbridge-id 3TOR1(config-rbridge-id-3)#
TOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Do youwant to continue? [y/n]:y…..TOR2#TOR2# conf tEntering configuration mode terminalTOR2(config)# rbridge-id 4rbridge-id 4TOR2(config-rbridge-id-4)#
VCS fabrics form adjacencies, called ISLs, with directly connected VCS-enabled neighboring switches.
ISLs can be separate links or Brocade-proprietary VCS fabric trunks. By default, all interfaces are
configured to detect ISLs. In this use case all ISLs are trunks. Verify that all the ISLs are up and have
formed adjacencies with neighboring devices.
Step 2: Verify the ISL status on all the devices.
EOR1EOR1# show fab isl
Rbridge-id: 1 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 15
53-1003534-01
-
8/15/2019 Brocade Storage
16/44
----------------------------------------------------------------------------------------------13 Te 1/0/14 17 Te 3/0/2 10:00:00:05:33:B8:00:18 20GYes"TOR1"14 Te 1/0/15 18 Te 4/0/3 10:00:00:05:33:EC:2A:D2 10GYes
"TOR2"EOR1#EOR2EOR2# show fabric isl
Rbridge-id: 2 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 2/0/14 17 Te 4/0/2 10:00:00:05:33:EC:2A:D2 20GYes"TOR2"14 Te 2/0/15 18 Te 3/0/3 10:00:00:05:33:B8:00:18 20GYes
"TOR1"EOR2#TOR1TOR1# show fabric isl
Rbridge-id: 3 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20GYes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20GYes"EOR2"
TOR1#TOR2TOR2# show fabric isl
Rbridge-id: 4 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20GYes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10GYes"EOR1"
TOR2#
Step 3: Verify that the VCS fabric is formed on all the devices.
EOR1EOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabric
Brocade VCS Storage Deployment Guide
16 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
17/44
Status HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7* 10.17.0.74 OnlineOnlineEOR1
2 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR1#
Note: “>10:00:00:05:33:86:D2:B6” listed above is the principal switch.EOR2EOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabric
Status HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6* 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR2#TOR1TOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18* 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2TOR1#TOR2TOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 17
53-1003534-01
-
8/15/2019 Brocade Storage
18/44
------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnline
EOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2* 10.17.0.112 OnlineOnlineTOR2TOR2#
Step 4: Configure FCoE on the edge interfaces.
TOR1. FCoE configTOR1# show running-config interface TenGigabitEthernet 3/0/10interface TenGigabitEthernet 3/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR1#
TOR2. CoE configTOR2# show running-config interface TenGigabitEthernet 4/0/10interface TenGigabitEthernet 4/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR2#
Currently, FCoE VLANs can be only 802.1Q VLANs. They cannot be classified or used as C-TAGs for
other VLAN classification. All tenant FCoE traffic uses the same default FCoE VLAN (1002) as in
previous Brocade NOS releases.
TOR1. FCoE login
TOR1# show fcoe fabric-map default ==================================================================================================Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]
Total number of Fabric Maps = 1TOR1#TOR1#TOR1#TOR1# show fcoe login =====================================================================================
========FCOE-Port Eth-port Device WWN Device MAC SessionMAC=============================================================================================Fcoe 1/3/1 Te 3/0/10 10:00:00:05:33:90:a7:00 00:05:33:90:a7:00 0e:fc:00:03:04:00
Total number of Logins = 1TOR1#
TOR1# show name-server detail PID: 030400
-
8/15/2019 Brocade Storage
19/44
Node Name: 20:00:00:05:33:90:A7:00 SCR: 3 FC4s: FCPPortSymb: [91] "Brocade-1020 | 3.2.1.0 | esxi51-0-99.englab.brocade.com |
VMware_ESXi_5.1.0_build-799733 | " NodeSymb: [58] "Brocade-1020 | 3.2.1.0 | esxi51-0-99.englab.brocade.com | " Fabric Port Name: 50:00:53:3B:80:05:30:4C
Permanent Port Name: 10:00:00:05:33:90:A7:00 Device type: Physical Initiator Interface: Fcoe 1/3/1 Physical Interface: Te 3/0/10 Share Area: Yes Redirect: NoPartial: No
TOR2. FCoE loginTOR2# show fcoe fabric-map default ==================================================================================================Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]
Total number of Fabric Maps = 1TOR2#TOR2# show name-server detailPID: 040400
-
8/15/2019 Brocade Storage
20/44
trunk-enable no shutdown!TOR2#
Step 6: Verify that the FC interfaces are UP and have formed an ISL with 8510.
NOTE
Refer to the configuration on the 8510 in the section "Configuring the storage network" later in this
document and make sure the configuration for the 8510 is also applied.
TOR1.TOR1# show interface Fibrechannel 3/0/1fibrechannel 3/0/1 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 0303c0PortIfId: 43011407PortWwn: 20:00:00:05:33:b8:00:18Distance: normal
Last clearing of show interface counters: 00:00:00Interrupts: 0 Link_failure: 0 Frjt: 0Unknown: 0 Loss_of_sync: 3 Fbsy: 0Lli: 130 Loss_of_sig: 5Proc_rqrd: 522359 Protocol_err: 0Timed_out: 0 Invalid_word: 107Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 5Suspended: 0 Lr_out: 0Parity_err: 0 Ols_in: 02_parity_err: 0 Ols_out: 2CMI_bus_err: 0
Rate info: Bandwidth: 8.00G Tx performance: 35.0KB/sec Rx performance:275.1MB/sec
TOR1#TOR1# show fabric isl
Rbridge-id: 3 #ISLs: 3
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------0 Fi 3/0/1 217 Fi 60/-/- 50:00:53:3C:01:20:0E:65 8GYes"fcr_fd_60"17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20GYes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20G
Yes"EOR2"
TOR1#
Login to TOR devices as root to verify that ISL ports have come up as E(Expansion)ports.
TOR1. E port VerificationTOR1:root>TOR1:root> switchshowName: sw0Type: 107.6State: Online
Brocade VCS Storage Deployment Guide
20 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
21/44
switchMode: NativeRole: Fabric SubordinateRbridge-id: 3switchId: fffc03WWN: 10:00:00:05:33:b8:00:18zoning: ON (cfg_vcs4)switchBeacon: OFF
FC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0303c0 id N8 Online FC E-Port 50:00:53:3c:01:20:0e:65"fcr_fd_60"(downstream)(Trunk Primary)
TOR2.TOR2# show interface Fibrechannel 4/0/2fibrechannel 4/0/2 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 040340PortIfId: 43011405PortWwn: 20:01:00:05:33:ec:2a:d2Distance: normal
Last clearing of show interface counters: 00:00:00Interrupts: 0 Link_failure: 0 Frjt: 0Unknown: 0 Loss_of_sync: 3 Fbsy: 0Lli: 90 Loss_of_sig: 5Proc_rqrd: 522692 Protocol_err: 0Timed_out: 0 Invalid_word: 9Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 4Suspended: 0 Lr_out: 1Parity_err: 0 Ols_in: 12_parity_err: 0 Ols_out: 1CMI_bus_err: 0
Rate info: Bandwidth: 8.00G
Tx performance: 40.4KB/sec Rx performance:288.2MB/secTOR2#TOR2# show fabric isl
Rbridge-id: 4 #ISLs: 3
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------1 Fi 4/0/2 2 Fi 61/-/- 50:00:53:3D:E0:5A:5E:65 8G Yes"fcr_fd_61"17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10G Yes
"EOR1"
TOR2#
Step 7: Login to the ToR devices as root to verify that the ISL ports have come up as E_ports.
TOR2. E port VerificationTOR2:root> switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric SubordinateRbridge-id: 4
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 21
53-1003534-01
-
8/15/2019 Brocade Storage
22/44
switchId: fffc04WWN: 10:00:00:05:33:ec:2a:d2zoning: ON (cfg_vcs4)switchBeacon: OFFFC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0403c0 id N8 No_Light FC1 1 040340 id N8 Online FC E-Port 50:00:53:3d:e0:5a:5e:65
"fcr_fd_61"(downstream)(Trunk Primary)
Step 8: Configure LSAN zoning in the VCS fabric.
LSAN Zone is the logical SAN zone which will be used to have zoning with multiple fabrics. In this
case, there are 2 fabrics "Server Network" and "Storage network". LSAN Zoning will be used.
Zoning config is a global config in the fabric, so this config can be applied on any one switch.
TOR1.TOR1# show running-config zoningzoning defined-configuration cfg cfg_vcs member-zone LSAN_T1
member-zone LSAN_T2 !zoning defined-configuration zone LSAN_T1 member-entry 50:06:01:6b:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:00!zoning defined-configuration zone LSAN_T2 member-entry 50:06:01:6a:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:01!zoning enabled-configuration cfg-name cfg_vcszoning enabled-configuration default-zone-access allaccesszoning enabled-configuration cfg-action cfg-saveTOR1#
TOR1# show zoning enabled-configuration zoning enabled-configuration cfg-name cfg_vcs
zoning enabled-configuration enabled-zone LSAN_T1
member-entry 50:06:01:6b:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:00
zoning enabled-configuration enabled-zone LSAN_T2 member-entry 50:06:01:6a:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:01TOR1#
Configuring the storage network
The following summarizes the steps involved in bringing up the storage network.
Step 1: Configure the LSAN zoning.
Step 2: Configure FC routing.
Step 3: Configure the EX_ports on the FC router.Step 4: Configure and verify other FC ports in the fabric.
To configure the storage network, complete the following steps.
Procedure
Step 1: Configure the LSAN zoning.
Configuring the storage network
22 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
23/44
Configure LSAN Zoning when there are multiple fabrics that require zoning (see the section Logical
SAN). Below is the running-config for the LSAN configuration in the current example.
DCX1.DCX1# zonecreate LSAN_T1 “50:06:01:6b:3e:a0:41:62; 10:00:00:05:33:90:a7:00”DCX1#cfgcreate cfg_DCX1,"LSAN_T1"DCX1#cfgenable cfg_DCX1
DCX:root> cfgshowDefined configuration: cfg: cfg_DCX1 LSAN_T1zone: LSAN_T1 50:06:01:6b:3e:a0:41:62; 10:00:00:05:33:90:a7:00Effective configuration: cfg: cfg_DCX1 zone: LSAN_T1 50:06:01:6b:3e:a0:41:62 10:00:00:05:33:90:a7:00
DCX2.DCX1# zonecreate LSAN_T2 “50:06:01:6a:3e:a0:41:62:62; 10:00:00:05:33:90:a7:01”DCX1#cfgcreate cfg_DCX2,"LSAN_T1"DCX1#cfgenable cfg_DCX2DCX:root> cfgshowDefined configuration: cfg: cfg_DCX2 LSAN_T2zone: LSAN_T2 50:06:01:6a:3e:a0:41:62; 10:00:00:05:33:90:a7:01
Effective configuration: cfg: cfg_DCX2 zone: LSAN_T2 50:06:01:6a:3e:a0:41:62 10:00:00:05:33:90:a7:01
Step 2: Configure FC routing.
Configure FC routing, which interconnects FC fabrics, whenever there are multiple fabrics in the SAN.
In this example, the 8510 has FC rouer blades and acts as the FC router.
DCX1. Enable FCRDCX1:root> fosconfig –enable fcrDCX:root> fosconfig --showFC Routing service: enabled iSCSI service: Service not supported on this PlatformiSNS client service: Service not supported on this PlatformVirtual Fabric: disabledEthernet Switch Service: disabledDCX:root>
DCX2. Enable FCRDCX2:root> fosconfig –enable fcrDCX2:root> fosconfig --showFC Routing service: enabled iSCSI service: Service not supported on this PlatformiSNS client service: Service not supported on this PlatformVirtual Fabric: disabledEthernet Switch Service: Service not supported on this PlatformDCX2:root>
Step 3: Configure EX_ports between both fabrics on the 8510.
DCX1. Enable FCRDCX1:root> portdisable 8/25DCX1:root>portcfgexport 8/25 -a 1 -f 101 -m 5 -d 60DCX1:root> portenable 8/25DCX:root> portcfgexport 8/25
Port 8/25 info Admin: enabled State: OKPid format: Not ApplicableOperate mode: Brocade NOSEdge Fabric ID: 101Front Domain ID: 60Front WWN: 50:00:53:3c:01:20:0e:65Principal Switch: 2Principal WWN: 10:00:00:05:33:86:d2:b6Fabric Parameters: Auto NegotiateR_A_TOV: 10000(N)E_D_TOV: 2000(N)Authentication Type: None
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 23
53-1003534-01
-
8/15/2019 Brocade Storage
24/44
DH Group: N/A
Hash Algorithm: N/AEdge fabric's primary wwn: N/AEdge fabric's version stamp: N/A
DCX:root>
DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0
Index Slot Port Address Media Speed State Proto217 8 25 01d900 id N8 Online FC EX-Port10:00:00:05:33:b8:00:18
"TOR1" (fabric id = 101 )(Trunk master)E-Port 50:00:53:3c:01:2e:0f:0c "fcr_xd_2_101" 218 8 26 01da00 id N8 In_Sync FC Disabled 219 8 27 01db00 id N8 No_Light FC
DCX2. Enable FCRDCX2:root> portdisable 2DCX2:root>portcfgexport 2 -a 1 -f 101 -m 5 -d 61DCX2:root> portenable 2DCX2:root> portcfgexport 2CAT5100:root> portcfgexport 2 Port 2 infoAdmin: enabledState: OKPid format: Not ApplicableOperate mode: Brocade NOSEdge Fabric ID: 101Front Domain ID: 61Front WWN: 50:00:53:3d:e0:5a:5e:65Principal Switch: 2Principal WWN: 10:00:00:05:33:86:d2:b6Fabric Parameters: Auto NegotiateR_A_TOV: 10000(N)E_D_TOV: 2000(N)Authentication Type: NoneDH Group: N/AHash Algorithm: N/AEdge fabric's primary wwn: N/AEdge fabric's version stamp: N/A
DCX2:root>DCX2:root>switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC F-Port 50:0a:09:85:8d:ae:5c:9c
1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC EX-Port 10:00:00:05:33:ec:2a:d2
Brocade VCS Storage Deployment Guide
24 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
25/44
"TOR2" (fabric id = 101 )(Trunk master)E-Port 50:00:53:3d:e0:5c:5f:08 "fcr_xd_1_101"
3 3 0a0300 id N8 No_Light FC
Step 4: Configure and verify all other FC ports in the fabric.
The following shows how to configure and verify the other FC ports in the fabric.
DCX1. Enable FCRDCX1:root> portenable 8/25DCX:root> DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0
Index Slot Port Address Media Speed State Proto
238 8 46 01ee00 id 8G Online FC F-Port 50:06:01:6b:3e:a0:41:62DCX2. Enable FCRDCX2:root> portenable 11DCX2:root> switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC F-Port 50:0a:09:85:8d:ae:5c:9c
1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC EX-Port 10:00:00:05:33:ec:2a:d2
"TOR2"(fabric id = 101 )(Trunk master)
E-Port 50:00:53:3d:e0:5c:5f:08 "fcr_xd_1_101"3 3 0a0300 id N8 No_Light FC4 4 0a0400 id N8 No_Light FC5 5 0a0500 id N8 No_Light FC6 6 0a0600 id N8 No_Light FC7 7 0a0700 id N8 No_Light FC8 8 0a0800 id N8 No_Light FC9 9 0a0900 id N8 No_Light FC
10 10 0a0a00 id N8 No_Light FC Disabled 11 11 0a0b00 id N8 Online FC F-Port 50:06:01:6a:3e:a0:41:62
Configuring LUNs on the storage device
This example demonstrates storage configuration with the EMC VNX5300 storage array with 15.5 TB
total hard disk space, and total usable storage of 15.4 TB.
The VNX5300 supports both file and block data services, optimized for virtual applications with VMware
vSphere or Microsoft Hyper-V.
To configure the LUNs on the VNX5300, complete the following steps
Configuring LUNs on the storage device
Brocade VCS Storage Deployment Guide 25
53-1003534-01
-
8/15/2019 Brocade Storage
26/44
-
8/15/2019 Brocade Storage
27/44
FIGURE 12 Creating the LUN
The system launches a wizard to guide you through the process of creating the LUN.
FIGURE 13 Creating the LUN Wizard
Step 3: Click Next and enter the size of the LUN in the page that is displayed.
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 27
53-1003534-01
-
8/15/2019 Brocade Storage
28/44
FIGURE 14 Sizing the LUN
Step 4: Choose an existing folder name for the LUN or create and assign a new folder.
FIGURE 15 Assigning a Folder to the LUN
Step 5: Click Finish to create the LUN.
Brocade VCS Storage Deployment Guide
28 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
29/44
FIGURE 16 Creating the LUN
Step 6: Confirm the LUN configuration details and note down unique LUN ID.
FIGURE 17 Confirming LUN Configuration
Note that unique LUN ID.
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 29
53-1003534-01
-
8/15/2019 Brocade Storage
30/44
-
8/15/2019 Brocade Storage
31/44
FIGURE 19 Login to the Server and Verify LUN Detection
Step 2: Verify that LUN ID matches with the device ID.
Step 3: Now assign this device to the host 10.17.0.95.
Step 4: To select the host, choose Edit Settings > Add > Harddisk > Raw device mappings.
FIGURE 20 Selecting the Host
Step 5: Choose the device to assign to this host.
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 31
53-1003534-01
-
8/15/2019 Brocade Storage
32/44
-
8/15/2019 Brocade Storage
33/44
Rack server and ToR connectivity to an FC SAN using an access
gateway
FIGURE 23 Rack server TOR connectivity to an FC SAN using an access gateway
NOTE
Figure 23 is the same as Figure 10 on page 14, but it has been repeated here to illustrate a different
use case.
FIGURE 24 Access gateway
Brocade Access Gateway (AG) is a feature provided by both NOS and FOS that lets you configure thefabric to handle additional N_Ports instead of domains. Switches in AG mode are logically transparent
to the host and the fabric. It increases the number of hosts that have access to the fabric without
increasing the number of switches in the fabric. This improves fabric scalability and simplifies
configuration and management in a large fabric by limiting the number of Domain IDs and ports.
In Figure 23 , ToR devices are configured as AGs. This causes the ToR device to appear like the
N_Port on an end node device in the SAN, so traffic is transmitted to and received only from the ToR
device, which then provides a gateway service to the hosts on the VCS NOS fabric to which it is
connected.
Rack server and ToR connectivity to an FC SAN using an access gateway
Brocade VCS Storage Deployment Guide 33
53-1003534-01
-
8/15/2019 Brocade Storage
34/44
The AG feature is supported only on the VDX 6730 or 6740 switches with FC ports from NOS Release
4.1 and later.
Enable the AG feature on VDX 6730/40 platforms to configure FC ports as N_Ports and map specific
virtual fabric (VF) ports to the N_Ports. This allows direct connection of hosts attached to VF_Ports on
the VDX switch with F_Ports on a FC fabric edge switch instead of through ISL connections from a
VDX 6730 to the FC router.
AG configuration changes are required on both the VCS fabric and the SAN. The rest of the
configuration is the same as the first example. The following is a summary of the special configuration
required.
Step 1: Configure the VCS server network.
Step 2: Configure the FC SAN.
Step 3: Configure LUNs on the storage devices.
Refer to the section Configuring LUNs on the storage device on page 25 for details.
Step 4: Map the LUNs on the server.
Refer to the section Mapping LUNs on the server on page 30 for details.
To configure the AG feature, complete the following steps.
Procedure
Step 1: Configure the VCS server network.
On the VCS fabric, configure the ToR VDX switch as an access gateway.
TOR1# ag enable ?Possible completions: | TOR1# ag enable
WARNING: Access Gateway mode changes the standard behavior of the switch.Please check Access Gateway Administrator's Guide before proceeding.
Enabling agmode will remove all the configuration data on the switchincluding zoning configuration and security database. Please backup yourconfiguration.
Enabling Access Gateway mode on the switch will reboot the switch. Do you want tocontinue?[y/n]:y
Repeat the configuration on the redundant ToR switch.
TOR2. Access gateway ConfigurationTOR2# ag enable
WARNING: Access Gateway mode changes the standard behavior of the switch.Please check Access Gateway Administrator's Guide before proceeding.Enabling agmode will remove all the configuration data on the switchincluding zoning configuration and security database. Please backup your
configuration.
Enabling Access Gateway mode on the switch will reboot the switch. Do you want tocontinue?[y/n]:y
The devices reboot and come up as an access gateway.
Verify that both ToR devices are running as access gateways.
TOR1. Access Gateway verificationTOR1# show agRbridge-ID 3:
Brocade VCS Storage Deployment Guide
34 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
35/44
----------------------------------------------------------------------------------------Name : TOR1NodeName : 10:00:00:05:33:b8:00:18Number of Ports : 76IP Address(es) : 10.17.0.111Firmware Version : v4.1.0_bld27
Number of N_Ports(Fi) : 1Number of VF_Ports : 1Policies Enabled : pgPersistent ALPA : DisabledPort Group information : PG_ID PG_Name PG_Mode PG_Members ---------------------------------------------------------- 0 pg0 lb 3/0/1, 3/0/2, 3/0/3, 3/0/4,
3/0/5, 3/0/6, 3/0/7, 3/0/8,3/0/9, 3/0/10, 3/0/11, 3/0/12,3/0/13, 3/0/14, 3/0/15, 3/0/16
----------------------------------------------------------Fabric Information : Attached Fabric Name N_Ports(Fi) ---------------------------------------------------------- 10:00:00:05:33:c0:12:00 3/0/1 ----------------------------------------------------------
N_Port(Fi) information :
Port PortID Attached PWWN IP_Addr VF_Ports ------------------------------------------------------------------------------------ Fi 3/0/1 0x01d900 20:d9:00:05:33:c0:12:00 10.17.0.131 1/3/1 ------------------------------------------------------------------------------------
VF_Port information : VF_Port Eth_Port PortID Attached PWWN N_Port(Fi) ------------------------------------------------------------------------------------ 1/3/1 Te 3/0/10 0x01d901 10:00:00:05:33:90:a7:00 3/0/1
------------------------------------------------------------------------------------TOR1#
Repeat this command on the second ToR switch.
TOR2.TOR2# show agRbridge-ID 4:----------------------------------------------------------------------------------------Name : TOR2NodeName : 10:00:00:05:33:ec:2a:d2Number of Ports : 76IP Address(es) : 10.17.0.112Firmware Version : v4.1.0_bld27Number of N_Ports(Fi) : 1Number of VF_Ports : 1Policies Enabled : pgPersistent ALPA : DisabledPort Group information : PG_ID PG_Name PG_Mode PG_Members ---------------------------------------------------------- 0 pg0 lb 4/0/1, 4/0/2, 4/0/3, 4/0/4,
4/0/5, 4/0/6, 4/0/7, 4/0/8,4/0/9, 4/0/10, 4/0/11, 4/0/12,
4/0/13, 4/0/14, 4/0/15, 4/0/16
----------------------------------------------------------Fabric Information : Attached Fabric Name N_Ports(Fi) ---------------------------------------------------------- 10:00:00:05:33:de:05:a5 4/0/2 ----------------------------------------------------------
N_Port(Fi) information : Port PortID Attached PWWN IP_Addr VF_Ports ------------------------------------------------------------------------------------ Fi 4/0/2 0x0a0200 20:02:00:05:33:de:05:a5 10.17.0.134 1/4/1
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 35
53-1003534-01
-
8/15/2019 Brocade Storage
36/44
------------------------------------------------------------------------------------
VF_Port information : VF_Port Eth_Port PortID Attached PWWN N_Port(Fi) ------------------------------------------------------------------------------------ 1/4/1 Te 4/0/10 0x0a0201 10:00:00:05:33:90:a7:01 4/0/2
------------------------------------------------------------------------------------TOR2#
NOTE
N_port and VF_Port information does not appear until SAN configuration is completed, as described
later.
Step 2: Configure the SAN.
This example uses a SAN an 8510 device facing the VCS, which must be configured as a regular
F_port as shown below.
DCX1. Enable FCRDCX1:root>portdisable 8/25DCX1:root> portcfgexport 8/25 -a 2DCX1:root>portenable 8/25DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0
Index Slot Port Address Media Speed State Proto217 8 25 01d900 id N8 Online FC F-Port 1 N Port + 1 NPIV publicDCX2. Enable FCRDCX2:root> portdisable 2DCX2:root> portcfgexport 2 -a 2DCX2:root>portenable 2DCX2:root> switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC
1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC F-Port 1 N Port + 1 NPIV public
Step 3: Configure LUNs on the storage devices.
Refer to the section Configuring LUNs on the storage device on page 25 for details.
Step 4: Map the LUNs on the server.
Refer to the section Mapping LUNs on the server on page 30 for details.
Brocade VCS Storage Deployment Guide
36 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
37/44
FC/FCoE storage connectivity from ToR rack server and ToR
connectivity to storage
FIGURE 25 FC/FCoE storage connectivity from ToR, Rack server and ToR connectivity to storage
With NOS release 4.1 and later, storage devices can be directly connected to FC interfaces of 6730.
With NOS release 5.0 and later, you can also use the VDX 6740.
In this use case there is no need for a separate SAN. VCS interconnects servers and using FCoE
interfaces and storage devices using FC interfaces. Because there is only one fabric, zoning
configuration is not required, although zoning is supported for access control.
The following are the steps required to implement this use case.
Step 1: Configure the VCS server network.
• Verify the ISL status on all the devices.
• Verify that VCS fabric is formed on all devices.
• Configure FCoE on the edge interfaces.
• Configure and verify FC interfaces on the TOR1 and TOR2 devices.
Step 2: Configure LUNs on the storage device.
Refer to the section Configuring LUNs on the storage device on page 25 for details.
Step 3: Map the LUNs on the server.
Refer to the section Mapping LUNs on the server on page 30 for details.
Procedure
To configure this example of converged network configuration, complete the following steps.
Step 1: Configure the VCS server network.
Bring up the VCS between TOR1, TOR2, EOR1, and EOR2 as shown below.
TOR1TOR2
FC/FCoE storage connectivity from ToR, rack server, and ToR connectivity to storage
Brocade VCS Storage Deployment Guide 37
53-1003534-01
-
8/15/2019 Brocade Storage
38/44
TOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y….TOR1#TOR1# conf tEntering configuration mode terminal
TOR1(config)# rbridge-id 3TOR1(config-rbridge-id-3)#
TOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Doyou wantto continue? [y/n]:y…..TOR2#TOR2# conf tEntering configuration mode terminalTOR2(config)# rbridge-id 4TOR2(config-rbridge-id-4)#
EOR1EOR2EOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y….EOR1#EOR1# conf tEntering configuration mode terminalEOR1(config)# rbridge-id 1EOR1(config-rbridge-id-3)#
EOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y…..EOR2#EOR2# conf tEntering configuration mode terminalEOR2(config)# rbridge-id 4EOR2(config-rbridge-id-4)#
Brocade VCS fabric forms adjacencies with directly connected Brocade VCS enabled neighboring
switches. These adjacencies are called ISLs. These ISLs can be separate links or Brocade proprietary
VCS Fabric Trunks. By default all interfaces are configured to detect ISL(Inter Switch Links) links/Trunks. In this use case all the ISLs are trunks. Verify if all the ISLs have come up and formed
adjacencies with neighboring devices.
Verify the ISL status on all the devices by entering the following commands:EOR1EOR1# show fab isl
Rbridge-id: 1 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 1/0/14 17 Te 3/0/2 10:00:00:05:33:B8:00:18 20G
Yes"TOR1"14 Te 1/0/15 18 Te 4/0/3 10:00:00:05:33:EC:2A:D2 10GYes"TOR2"
EOR1#EOR2EOR2# show fabric isl
Rbridge-id: 2 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW
Brocade VCS Storage Deployment Guide
38 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
39/44
Trunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 2/0/14 17 Te 4/0/2 10:00:00:05:33:EC:2A:D2 20G Yes"TOR2"
14 Te 2/0/15 18 Te 3/0/3 10:00:00:05:33:B8:00:18 20G Yes"TOR1"
EOR2#TOR1TOR1# show fabric isl
Rbridge-id: 3 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20G Yes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"
TOR1#TOR2TOR2# show fabric isl
Rbridge-id: 4 #ISLs: 2
Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10G Yes"EOR1"
TOR2#
Verify that VCS fabric is formed on all the devices by entering the following commands:
EOR1EOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7* 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 Online
OnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR1#
Note: “>10:00:00:05:33:86:D2:B6” listed above is the principal switch.EOR2EOR2# show vcsConfig Mode : Local-Only
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 39
53-1003534-01
-
8/15/2019 Brocade Storage
40/44
VCS Mode : Fabric Cluster VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName-------------------------------------------------------------------------------------
-------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6* 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR2#TOR1TOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18* 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineOR2TOR1#TOR2TOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster
VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2* 10.17.0.112 OnlineOnlineTOR2TOR2#
Configure FCoE on the edge interfaces by entering the following commands:
TOR1. FCoE configTOR1# show running-config interface TenGigabitEthernet 3/0/10interface TenGigabitEthernet 3/0/10 fabric isl enable
Brocade VCS Storage Deployment Guide
40 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
41/44
fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR1#TOR2. CoE config
TOR2# show running-config interface TenGigabitEthernet 4/0/10interface TenGigabitEthernet 4/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR2#
Currently, FCoE VLANs can be only 802.1Q VLANs. They cannot be classified or used as C-TAGs for
other VLAN classification. All tenant FCoE traffic is carried on the same default FCoE VLAN (1002) as
in previous NOS releases.
TOR1. FCoE loginTOR1# show fcoe fabric-map default==================================================================================================
Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]
Total number of Fabric Maps = 1TOR1#TOR1#TOR1#TOR1# show fcoe login =============================================================================================FCOE-Port Eth-port Device WWN Device MAC SessionMAC
=============================================================================================Fcoe 1/3/1 Te 3/0/10 10:00:00:05:33:90:a7:00 00:05:33:90:a7:00 0e:fc:00:03:04:00
Total number of Logins = 1TOR1#
TOR1# show name-server detailPID: 030400
-
8/15/2019 Brocade Storage
42/44
Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]
Total number of Fabric Maps = 1TOR2#TOR2# show name-server detailPID: 040400 switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric Subordinate
Brocade VCS Storage Deployment Guide
42 Brocade VCS Storage Deployment Guide
53-1003534-01
-
8/15/2019 Brocade Storage
43/44
Rbridge-id: 3switchId: fffc03WWN: 10:00:00:05:33:b8:00:18zoning: OFFswitchBeacon: OFFFC Router: OFFFC Router BB Fabric ID: 128
Address Mode: 0Index Port Address Media Speed State Proto============================================== 0 0 0303c0 id N8 Online FC E-Port 50:00:53:3c:01:20:0e:65"fcr_fd_60"(downstream)(Trunk Primary)1 1 030340 id N8 No_Sync FC Disabled (Persistent)2 2 0302c0 -- N8 No_Module FC3 3 030240 id N8 Online FC F-Port 50:06:01:69:3e:a0:41:62 4 4 030380 -- N8 No_Module FC5 5 030300 -- N8 No_Module FC6 6 030280 -- N8 No_Module FC7 7 030200 -- N8 No_Module FC8 8 0301c0 -- N8 No_Module FC9 9 030140 -- N8 No_Module FC10 10 0300c0 -- N8 No_Module FC11 11 030040 -- N8 No_Module FC
TOR2. FCoE loginTOR2# show running-config interface FibreChannel 4/0/4interface FibreChannel 4/0/4 desire-distance 0 no isl-r_rdy trunk-enable no shutdown!TOR2# show interface Fibrechannel 4/0/4fibrechannel 4/0/4 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 041400PortIfId: 43011401PortWwn: 20:03:00:05:33:ec:2a:d2Distance: normal
Last clearing of show interface counters: 00:00:00
Interrupts: 0 Link_failure: 2 Frjt: 0Unknown: 0 Loss_of_sync: 7 Fbsy: 0Lli: 551 Loss_of_sig: 13Proc_rqrd: 44 Protocol_err: 0Timed_out: 0 Invalid_word: 30Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 3Suspended: 0 Lr_out: 3Parity_err: 0 Ols_in: 32_parity_err: 0 Ols_out: 3CMI_bus_err: 0
Rate info: Bandwidth: 8.00G Tx performance: 0 B/sec Rx performance: 0 B/secTOR2#
Also login as root and verify that FC interface has detected the peers wwn.
TOR2:root> switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric SubordinateRbridge-id: 4switchId: fffc04WWN: 10:00:00:05:33:ec:2a:d2zoning: OFFswitchBeacon: OFF
Brocade VCS Storage Deployment Guide
Brocade VCS Storage Deployment Guide 43
53-1003534-01
-
8/15/2019 Brocade Storage
44/44
FC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0
Index Port Address Media Speed State Proto============================================== 0 0 0403c0 id N8 Online FC E-Port 50:00:53:3d:e0:5a:5e:65
"fcr_fd_61"(downstream)(Trunk Primary)1 1 040340 id N8 Online FC E-Port (Trunk port, Primary is Port
0 ) 2 2 0402c0 -- N8 No_Module FC Disabled (Persistent)3 3 041400 id N8 Online FC F-Port 50:06:01:6f:3e:a0:41:62
Step 2: Configure LUNs on the storage devices.
Refer to the section Configuring LUNs on the storage device on page 25 for details.
Step 3: Map the LUNs on the server.
Refer to the section Mapping LUNs on the server on page 30 for details.
Brocade VCS Storage Deployment Guide