1 © 2004 cisco systems, inc. all rights reserved. iscsi overview ip storage networking fcip/iscsi...
TRANSCRIPT
1© 2004 Cisco Systems, Inc. All rights reserved.iSCSI Overview
IP Storage NetworkingFCIP/iSCSI
Steve TegelerStorage Networking TeamNorthwest Territory425/[email protected]
222© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• IP Storage Networking
• FCIPWrite Acceleration, Compression, IPSec, SAN Extension Tuner
• FCIP Wizard
• iSCSIWhat, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
333© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
444© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
The Old Storage Environment
• Direct Attached Storage (DAS)
• Storage is captive ‘behind’ the server
• Server CPU must handle user I/O requests, but also:
User-database inquiries
User file/print serving
Data-integrity checking
Communication with other devices
• Data access is file system and platform dependant
• Costly to scale; complex to manage
FC
Clients
SCSIFC
Direct-Attached Storage (DAS)
Servers
Win2k Linux Win2k Linux Unix
IP Network
555© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
The SCSI I/O Channel
• SCSI is the dominant protocol used to communicate between servers and storage devices in open system
• SCSI I/O channel is a half-duplex pipe for SCSI CDBs and data
• Parallel bus evolutionBus width: 8, 16 bitsBus speed: 5–80 MhzThroughput: 5–320 MBpsDevices/bus: 2–16 devicesCable length: 1.5m–25m
• A network approach can scale the I/O channel in many areas (length, devices, speed)
SCSI CDB: SCSI Command Descriptor Block Used to Relay SCSI Commands, Parameters, and Status between SCSI Initiators and SCSI Targets; Typically 6, 10, or 12 Byte Block
SCSI AdapterSCSI Adapter
Applications
File SystemFile System
Block DeviceBlock Device
SCSI GenericSCSI Generic
TCP/IPStack
TCP/IPStack
NICDriverNIC
Driver Adapter DriverAdapter Driver
Half-DuplexSCSII/O Channel
SCSIInitiator
SCSITarget
SCSI
RawRaw
EthernetNIC
EthernetNIC
Ethernet
666© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Networking the I/O Channel
• Same SCSI protocol (SCSI-3) carried over a network transport layer via serial implementation
• Transport must not jeopardize SCSI payload (security, integrity, latency)
• Two primary transports to choose from today: Fibre Channel and IP
• A networked I/O channel allows for multiple improvements:
Distance limitations greatly increased
High number of addressable devices
Initiator
Target and LUNsN
etw
ork
edI/
O C
han
nel Channel
Controller
SCSI
Host System
Network
777© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Fibre Channel Networking
• Very common method for networking SCSI
• Fibre Channel provides high-speed transport for SCSI payload
• Fibre Channel SAN overcomes many shortcomings of DAS including:
Addressing for up to 16-million nodes (24 bits)
Loop (shared) and Fabric (switched) transport
Speeds of 100 or 200 MBps (1 or 2 Gbps)
Distance of up to 10km (without extenders)
Can utilize CWDM or DWDM for over 10km
Support for multiple protocols
• Combines best attributes of a channel and a network
FibreChannel HBA
Fibre Channel Fabric
Initiator
SCSI
Host System
Target
888© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
IP: An Alternate I/O Transport
• Viable transport for I/O traffic
• Not necessarily for long-haul I/O only
• Similar characteristics to Fibre Channel:
Addressing for close to 4 billion nodes (IPv4)
Primarily a switched transport (with routing)
Ethernet speeds of 10/100 Mbps or 1/10 Gbps or various WAN speeds
Support for multiple high-level protocols
• Cost and manageability advantages with IP
• IP knowledge base widespread in industry
IP “ChannelAdapter”
Target
SCSI
Host System
Initiator
IP Network
999© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
IP Storage Networking
• IP storage networking provides solution to carry storage traffic within IP
• Uses TCP: a reliable transport for delivery• Applicable to local data center and long-haul applications• Two primary protocols:
iSCSI—Internet-SCSI—used to transport SCSI CDBs and data within TCP/IP connections
FCIP—Fibre-Channel-over-IP—used to transport Fibre Channel frames within TCP/IP connections—any FC frame—not just SCSI
IP TCPTCP FCIPFCIP FCFC SCSI Data
IP TCPTCP iSCSIiSCSI SCSI Data
101010© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
FCIP – Extending your FC SAN
101010© 2004, Cisco Systems, Inc. All rights reserved.
111111© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Fibre Channel over IP (FCIP)Point to Point
FCIP – Fibre Channel over Internet Protocol
The encapsulation of Fibre Channel frames into IP packets and tunneling through an existing TCP/IP network infrastructure, in order to connect geographically distant
islands
LAN/MAN/WAN
FCIP Tunnel SessionFCIP Tunnel Session
FC DiskFC Disk
SAN SAN
Ethernet Catalyst Switches & Routers
Optical Extension Metro DWDM
& CWDM
IPSIPS
Sync or AsyncReplication
E-port E-port
FCIP tunnels can be thought of as
ISL’s with Latency
121212© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI
121212© 2004, Cisco Systems, Inc. All rights reserved.
131313© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Audience Poll
• Who has a FC Network today?
• Who has deployed iSCSI,
• Array based, or gateway based?
141414© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
What is iSCSI?
• A SCSI transport protocol that operates over TCP/IPEncapsulates SCSI CDBs (operational commands: e.g. read or write) and data into TCP/IP byte streams
Allows IP hosts to access IP-based SCSI targets (either natively or via iSCSI to FC Gateways)
• Standards statusRFC 3720 on iSCSI
Collection of RFCs describing iSCSI
RFC 3347—iSCSI Requirements
RFC 3721—iSCSI Naming and Discover
RFC 3723—iSCSI Security
• Broad industry supportServer vendors now publishing own supported iSCSI drivers
Native iSCSI storage arrays now appearing
151515© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
SCSI Block CommandsSCSI Block CommandsSCSI StreamSCSI StreamCommandsCommands
Parallel Parallel SCSI TransportSCSI Transport
SCSI Applications (File Systems, Databases)
Parallel SCSIParallel SCSIInterfacesInterfaces
SCSIDevice-TypeCommands
SCSIGeneric
Commands
SCSITransportProtocols
Layer 3 Network
Transport
Layer 2Network Fibre ChannelFibre Channel Ethernet, PPP, HDLC…Ethernet, PPP, HDLC…
Other SCSI CommandsOther SCSI Commands
IPIP
TCPTCP
SCSI Commands, Data, and Status
Recap SCSI Architectural Model Transports
FCPFCPSCSI over FCSCSI over FC
iSCSIiSCSISCSI over TCP/IPSCSI over TCP/IP
161616© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Why - iSCSI vs. FC It’s all about the $$$
• Leverage IP infrastructure for storage connectivity
• Low-cost complement to FC SAN, provides additional resource consolidation
1000BaseT NIC + GigE Port: Roughly $100 + $300 = $400
FC HBA+ FC Port: Roughly $1000 + $1000 = $2000
• Secure connectivity via CHAP-based authentication
• Transparent iSCSI routing gives iSCSI hosts a pWWNUses controller-based LUN masking or MDS-based virtual
targets for resource provisioning
Uses zoning for device connectivity
• iSCSI driver (free) works with any Ethernet NICTOE only necessary with processor-bound servers
• iSCSI is an industry-supported IETF standard
• Many O/S vendors providing iSCSI initiator (MS, HP, Novell, Linux), others provided by Cisco
iSCSI-enabled Hosts
FC Disk Array
CatalystEthernetSwitch
IPNetwork
Cisco MDS 9000 with
IP Services Module
iSCSI
iSCSI
iSCSI
FC Servers
FC Tape Library
171717© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI for Storage Consolidation
• IP access to open systems iSCSI and Fibre Channel storage
• iSCSI driver is loaded onto hosts on Ethernet network
• Able to consolidate servers via iSCSI onto existing storage arrays
• Able to build Ethernet-based SANs using iSCSI arrays
• Storage assigned on a LUN-by-LUN basis at iSCSI router
iSCSI-EnabledHosts (Initiators)
iSCSI Array
(Target)
StoragePool (Target)
iSCSIGateway
IPNetwork
FCFabric
FCFabric
FC HBA-Attached
Host (Initiator)
iSCSI
iSCSI
Logical Unit Number (LUN): A Field within SCSI Containing up to 64 Bits that Identifies the Logically Addressable Unit within a Target SCSI Device
iSCSIiSCSI
181818© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI for Remote Block Access
• Block access to remote storage over IP
• Application must tolerate latency for long distances
• Metro Ethernet services offer lower-latency transport alternative
• Remote backup over IP WAN
• Centralized management from centralized storage
iSCSI-EnabledHost
Remote Mirrors
IPWAN
StoragePool
FCFabric
FCFabric
iSCSI Device
Site A
Site B
iSCSI
191919© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
How - iSCSI Architecture: Software Driver
iSCSI GW Module
SCSI DriverSCSI Driver TCP/IP DriverTCP/IP Driver
FC HBAFC HBA GigE NICGigE NIC
NICNIC SCSI AdapterSCSI Adapter
File SystemFile System
Block DeviceBlock Device
SCSI GenericSCSI Generic
TCP/IPStack
TCP/IPStack
NICDriverNIC
Driver Adapter DriverAdapter Driver
iSCSI
iSCSI GW Device
Host
iSCSI Host Driver
Conventional SCSI Path
iSCSI Path
IP Network
Fibre ChannelFibre Channel
Applications
202020© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
OS Support
• Many operating systems supported via Cisco drivers and/or from OS vendor
Cisco provides full-driver suite
Solaris 2.6 (EOL),7,8,9
Linux-based on 2.6 kernel
Win 2000 with SP2 or later
Windows XP Pro
WinNT 4.0 with SP6A
HP/UX 10.2, 11.0
AIX 4.3.3, 5.1, 5.2
OS vendors support native iSCSI drivers
Windows *native* Win 2000, XP, 2003 support
HP *native* HP/UX 11i support
IBM *native* AIX 5.x support
Novell Netware *native* support
Solaris 10 (March 2005)
Linux (RedHat Suse)
NIC AdapterNIC Adapter SCSI Adapter
SCSI Adapter
Adapter Driver
Adapter Driver
iSCSI
SCSI GenericSCSI Generic
Applications
File SystemFile System
Block DeviceBlock Device
TCP/IP StackTCP/IP Stack
NIC DriverNIC Driver
iSCSISoftware
Driver
212121© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI HBAs and TCP Offload Engines (TOEs)
• Offloads TCP and, optionally, iSCSI processing into hardware
• Relieves host CPU from:
TCP processing—16-bit checksum per packet
iSCSI—optional 32-bit header and data digests (CRC32C)
TCP Offload
iSCSI and TCP Offload
File SystemFile System
Block DeviceBlock Device
SCSI GenericSCSI Generic
HBADriverHBA
Driver
TCP/IPStack
TCP/IPStack
NICDriverNIC
Driver
iSCSIDriver
TOEDriverTOE
Driver
SCSI AdapterSCSI Adapter
Adapter Adapter DriverDriver
TCP/IPStack
TCP/IPStack
iSCSITCP/IPStack
TCP/IPStack
Applications
222222© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
232323© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Example performance impact on CPU util % FC vs. iSCSI TOE vs. iSCSI SW Driver
CP
U %
Throughput MB/s
FC HBA
iSCSI TOE
iSCSI SW Driver
15-35MB/sInflection point determined by
system resources (CPU/Memory)
?
242424© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
252525© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI Naming
• Initiator and target require iSCSI names
Name is location independent
iSCSI node name = SCSI device name of iSCSI device
Associated with iSCSI nodes, NOT adapters
Up to 255-byte displayable/human readable string (UTF-8 encoding)
Use SLP (Service Location Protocol) V2, iSNS, or query target for names (SendTargets)
• Two iSCSI name types:
iqn—iSCSI qualified name
eui—Extended Unique Identifier (IEEE EUI-64—also used for FC WWNs)
262626© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
–– –
iSCSI Name Structure
– Unique String
iqn.1987-05.com.cisco.1234abcdef987601267da232.bettyiqn.2001-04.com.acme.storage.tape.sys1.xyz
Type DateOrganization
Naming AuthoritySubgroup Naming Authority or
String Defined by Organization Naming Authorityiqn
eui
Date = yyyy-mm When Domain Acquired
Reversed Domain Name
–Type EUI-64 Identifier (ASCII Encoded Hexadecimal)
eui.02004567a425678d
Type
272727© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI Connectivity
• iSCSI Initiator knows IP and IQN
• FC Target knows WWN and FCID
iSCSIHBA
HBA
iSCSIHBA
HBA FC
Fibre ChannelFabric
iqn.host-3IP-10.1.1.4
iqn.host-1IP-10.1.1.2
pWWN – P6nWWN –N6
FCID – XXXX06
pWWN – P5nWWN – N5
FCID –XXXX05
IP-10.1.1.1IPNetwork
iSCSIHBA
HBA
iqn.host-2IP-10.1.1.3
282828© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
iSCSI Mapping to a WWN
• Each iSCSI Initiator gets a unique WWN and FCID
iSCSIHBA
HBA
iSCSIHBA
HBA FC
Fibre ChannelFabric
iqn.host-3IP-10.1.1.4
iqn.host-1IP-10.1.1.2
pWWN – P6nWWN –N6
FCID – XXXX06
pWWN – P5nWWN – N5
FCID –XXXX05
IP-10.1.1.1
iSCSIHBA
HBA
iqn.host-2IP-10.1.1.3
pWWN – P2nWWN- N2
FCID XXXX02
pWWN – P4nWWN- N4
FCID XXXX04
pWWN – P3nWWN – N3
FCID XXXX03
292929© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
303030© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Considerations when building an iSCSI Fabric
• iSCSI fabric topologyEthernet fabric topology
• iSCSI fabric scalabilityTrunking
Port channeling
• iSCSI fabric availabilityVRRP
• iSCSI fabric securityAuthentication and binding
• iSCSI fabric manageabilityiSCSI identity and management
ScalabilityAvailability
SecurityManageability
iSCSIClients
END
TO
END
END
TO
END
Shared Storage Pool
iSCSI iSCSI
iSCSI iSCSI iSCSI
iSCSI iSCSI
iSCSI iSCSI iSCSI
IPSIPSIPSIPS
313131© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Dedicated IP Storage Network
• Separate logical IP network but not necessarily separate physical network
• Can use a VLAN of existing Ethernet network
• Recommend use of dedicated NIC on host for iSCSI
• Minimized potential for bandwidth contention
iSCSI-EnabledHosts
Storage Pool
iSCSIRouters
CatalystSwitches
DedicatedIP Storage Network
FCFabric
FCFabric
Clients
Front-Side IP Network
FC-Attached Hosts with HBAs
iSCSI iSCSI iSCSI iSCSI
333333© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
IP Network Security Techniques
• FirewallStandalone or intelligent firewall service module
Allow well-known TCP port 3260 for iSCSI
• IPSec VPN VPN tunnel for iSCSI remote access
• Access Control List (ACL)
• VLAN and PVLANSubinterface implementation on iSCSI
Separated VLAN for iSCSI
• Port securityAllow, block, or restrain access to Ethernet based on MAC address
343434© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
What is iSNS?
Internet Storage Name Service (iSNS) is a name registration service for IP storage devices:
Analogous to FCNS and DNS
Provides centralized management capabilities
iSNS supports:
Target device discovery
Discovery Domains (similar to zones)
Authentication
State change notification
Supports iSCSI and iFCP
353535© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
What is iSNS? (cont.)
IP
FC
iFCPGateway
IP
IP
iSNS server
FC
FC
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
IP
IP
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
iSCSIiSCSI
FC
iSNSiSNS
363636© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
373737© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Summary
• Leverages the existing IP infrastructure
Hence the intelligence, capacity, and best practice design can be leveraged in the iscsi infrastructure
• Complementary to FC yet represents a low-cost transport choice
• Midrange applications connectivity
• Midrange server connectivity with blade server integration as new system candidate
• Potential long-distance SAN transport
383838© 2004 Cisco Systems, Inc. All rights reserved.
OPT-20539761_05_2004_c2
Reference Materials
• http://www.t10.org/
• http://www.t11.org/index.htm
• http://www.ietf.org/rfc.html on RFC 3720
• http://www.cisco.com/en/US/partner/products/hw/ps4159/index.html
• http://www.lightreading.com/webinar_archive_home.asp?webinar_id=27003