networking technologies: concepts in internal and external

122
EDUCATION Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved. Networking Technologies: Concepts in Internal and External Networked Storage Howard Goldstein Howard Goldstein Associates, Inc. [email protected]

Upload: networksguy

Post on 28-Jan-2015

115 views

Category:

Documents


3 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Networking Technologies: Concepts in Internal and External

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage©Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Networking Technologies: Concepts in Internal and External Networked Storage

Howard GoldsteinHoward Goldstein Associates, [email protected]

Page 2: Networking Technologies: Concepts in Internal and External

2

EDUCATION

Storage Networking Concepts

• Objectives:– Identify what networking is and how it relates to

storage– Examine storage networking technology similarities

and differences– Clarify and explain the language of networking:

Terminology– Uniquely inform and hopefully entertain

NetworkingTechnologies

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 3: Networking Technologies: Concepts in Internal and External

3

EDUCATION

Outline

NetworkingTechnologies

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 4: Networking Technologies: Concepts in Internal and External

4

EDUCATION

Outline• Network vs. Storage• ANs and ASes• Perspectives• Virtual SCSI Cables• Port vs. PHY• Switch vs. Router vs.

Gateway• Circuit Switch vs. Packet

Switch• Connection-Oriented vs.

Connectionless

• Layered Protocols• Flow Control • Payload Integrity• Names vs. Addresses• Segmentation,

Convergence, Reassembly & Offload

• Bit Rate, Bandwidth, Throughput

• Semantic Analysm• Future & Conclusions

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 5: Networking Technologies: Concepts in Internal and External

5

EDUCATION

Network Vs. Storage

Network Considerations Storage ConsiderationsInformation movement Information repositoryData over Distance Data over TimeConfiguring and segmentingnetwork topologies

Formatting and partitioning hard disks, Creating logical volumes

Hubs, Bridges, Switches, Routers, Gateways

JBOD, RAID, Disk Controllers, Tape Controllers

Client / Server Applications Initiator / Target FunctionsEmail, Web Browsing, File Mgt,Network Interface

Backup, Archive, Mirror, Block Mgt, I/O Interfaces

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 6: Networking Technologies: Concepts in Internal and External

6

EDUCATION

Network Vs. Storage

Network Considerations Storage ConsiderationsSpeed, Solid-State Capacity, Mechanical MovementMedia – Copper, Fiber, Wireless Media – Disk, Tape, PaperBits in Frames, CRC error checks every frame

Bytes in Blocks, Parity check every byte

Buffers and Link transfer rates –Kbps, Mbps, Gbps

Cache and Disk/Tape I/O interface transfer rates - MBps

Interconnectivity network and device latencies - bottlenecks

Disk/Tape seek times and latencies – ms, us, ns

Applications – HTTP, NFS, SMTPTransport Protocols –TCP/IP/Ethernet

Logical Device Protocols – SCSI, ESCON, IDE/ATAPhysical Protocols – Parallel SCSI, Fibre Channel, iSCSI/TCP/IP/Ethernet

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 7: Networking Technologies: Concepts in Internal and External

7

EDUCATION

Storage Data Units of Measure

Unit of Measure

Abbreviation Approximate Size

Byte Byte 8 bitsBlock Blk 512 BytesKilobyte KB 1,024 Bytes

Megabyte MB 1 Million Bytes

Gigabyte GB 1,000 MBTerabyte TB 1,000 GBPetabyte PB 1,000 TBExabyte EB 1 Million TB

Zetabyte ZB 1,000 EBYottabyte YB 1,000 ZB

How long would it take someone to download a one-yottabyte file using a 56 Kbps modem?

5.5 trillion years.

ReallyReally Big Big NumbersNumbers

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 8: Networking Technologies: Concepts in Internal and External

8

EDUCATION

Network Data Units of Measure

1 1

Unit of Measure

Power of 10

Number

milli - 3 One thousandth

micro - 6 One millionth

nano - 9 One billionth

pico - 12 One trillionth

femto - 15 One quadrillionth

atto - 18 One quintillionth

zepto - 21 One sextillionth

yocto - 24 One septillionth

Really Really Small Small NumbersNumbers

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 9: Networking Technologies: Concepts in Internal and External

9

EDUCATION

Basic Storage Network Technology

Direct-Attached Storage (DAS)

Application

File System

Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 10: Networking Technologies: Concepts in Internal and External

10

EDUCATION

Basic Storage Network Technology

Direct-Attached Storage (DAS)

Application

File System

Storage

Direct-Attached Storage (DAS)

Application

File System

Storage

Direct-Attached Storage (DAS)

Application

File System

Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 11: Networking Technologies: Concepts in Internal and External

11

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Application

File System

Storage

Network

Application

File System

Storage

Direct-Attached Storage (DAS)

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 12: Networking Technologies: Concepts in Internal and External

12

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Application

File System

Storage

Application Application

Network

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 13: Networking Technologies: Concepts in Internal and External

13

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Storage Area Network (SAN)

Application

File System

Storage

Network

Application

File System

Storage

Direct-Attached Storage (DAS)

Application

File SystemNetwork

Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 14: Networking Technologies: Concepts in Internal and External

14

EDUCATION

Basic Storage Network Technology

Storage Area Network (SAN)

Application

File System

Storage

Network

Storage Storage

Application

File System

Application

File System

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 15: Networking Technologies: Concepts in Internal and External

15

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Storage Area Network (SAN)

Application

File System

Storage

Network

Application

File System

Storage

Direct-Attached Storage (DAS)

Application

File System

Network

Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 16: Networking Technologies: Concepts in Internal and External

16

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Storage Area Network (SAN)

Application

Storage

Application

File System

Network

Storage

Network

File System

Page 17: Networking Technologies: Concepts in Internal and External

17

EDUCATION

Basic Storage Network Technology

Network-Attached Storage (NAS)

Storage Area Network (SAN)

Application

File System

Storage

Network

Application

File SystemNetwork

Storage

NAS & SAN

Page 18: Networking Technologies: Concepts in Internal and External

18

EDUCATION

Application

File System

Storage

File Request File RequestFile

Block Request Block RequestBlocks

Blocks

File

DAS

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 19: Networking Technologies: Concepts in Internal and External

19

EDUCATION

Network Attached Storage

NAS

•Messages

•Files

NFS/CIFS Client

NFS/CIFS Server

LAN NAS

File

File Request

Blocks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 20: Networking Technologies: Concepts in Internal and External

20

EDUCATION

Storage Area Network

SERVER

NASGateway

SAN

Storage

• Commands

• Blocks

• Control

• Status

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 21: Networking Technologies: Concepts in Internal and External

21

EDUCATION

Storage Area Network – “External SAN”

SERVER

SAN

SCSI Initiator Client

SCSI Target & LUN Server

Read Command Request

Blocks

Status

“External SAN”

File

File RequestApplication

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 22: Networking Technologies: Concepts in Internal and External

22

EDUCATION

NetworkNetwork

Network

LAN MAN WAN

Redundant Array of Independent Disks

Disk Drives

ControllerA

ControllerB

Network“SmAN”

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 23: Networking Technologies: Concepts in Internal and External

23

EDUCATION

LAN/SAN Benefits

• Greater Asset Utilization

• Expanded connectivity• Port consolidation• Extended distance• Higher bandwidth• Potential performance

improvement

LAN

SAN

FC IP

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 24: Networking Technologies: Concepts in Internal and External

24

EDUCATION

LAN & SAN Information Flow Requirements

LAN SAN

Ethernet

IP

TCP, UDP

iSCSI, HTTP, SMTP, NFS, FTP, DNS

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 25: Networking Technologies: Concepts in Internal and External

25

EDUCATION

“External SAN” I/O Request to Storage Controller

Controller

Controller

SAN

External Read Command Request

Disk

Disk

Disk

Disk

DiskDisk

External SCSI Target & LUN

Blocks BlocksStatus

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 26: Networking Technologies: Concepts in Internal and External

26

EDUCATION

“Internal SAN” I/O Requests in Storage Controller

Controller 2

Controller 1 Controller 1 Internal SCSI Initiator Client

Internal SCSI Target & LUN (Disk Drive)

Controller 1 Internal Read Command Request

Blocks

Status

“Internal SAN”

RAID

Controller 1 Internal Read Command Request

Internal SCSI Target & LUN (Disk Drive)

External Read Command Request

Disk

Disk

Disk

Disk

DiskDisk

Status

Blocks

External SCSI Target & LUN

Blocks Blocks

Status

FC Loop

SwitchSAN

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 27: Networking Technologies: Concepts in Internal and External

27

EDUCATION

Controller 2

Controller 1 Internal SCSI Initiator Client

Internal SCSI Target & LUN (Disk Drive)

Controller 1 Internal Read Command Request

Blocks

Status

“Internal SAN”

RAID

Controller 1 Internal Read Command Request

Internal SCSI Target & LUN (Disk Drive)

External Read Command Request

Disk

Disk

Disk

Disk

DiskDisk

Status

Blocks

External SCSI Target & LUN

Blocks Blocks

Status

“Internal SAN” I/O Requests in Storage Controller

FC Loop

Switch

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 28: Networking Technologies: Concepts in Internal and External

28

EDUCATION

Controller

Controller Internal SCSI Initiator Client

Internal SCSI Target & LUN (Disk Drive)

Internal Read Command Request

Blocks

Status

“Internal SAN”

RAID

Internal Read Command Request

Internal SCSI Target & LUN (Disk Drive)

External Read Command Request

Disk

Disk

Disk

Disk

DiskDisk

Status

Blocks

External SCSI Target & LUN

Blocks Blocks

Status

“Internal SAN” I/O Requests in Storage Controller

SAS Expander

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 29: Networking Technologies: Concepts in Internal and External

29

EDUCATION

Virtual SCSI Cables on SCSI Bus

Target

Logical Units0 1 2 62 63

PeripheralController

SCSI Bus

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Adapter

Adapter

Target

Initiator

P-SCSI

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 30: Networking Technologies: Concepts in Internal and External

30

EDUCATION

Virtual SCSI Cables on SCSI Bus

Target

Logical Units0 1 2 62 63

PeripheralController

SCSI Bus

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Adapter

Adapter

Target

Initiator

SCSI Bus: Arbitrate ID, Select ID, Identify LUN

Blocks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 31: Networking Technologies: Concepts in Internal and External

31

EDUCATION

Virtual SCSI Cables Fibre Channel

FC: Fabric Login, N_Port Login, FC-4 Login

FC SAN

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Logical Units0 1 2 62 63

PeripheralController

Adapter

Target

Logical Units0 1 2 62 63

PeripheralControllerAdapter

Target

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 32: Networking Technologies: Concepts in Internal and External

32

EDUCATION

Logical Units0 1 2 62 63

PeripheralController

Adapter

Target

Logical Units0 1 2 62 63

PeripheralControllerAdapter

Target

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Virtual SCSI Cables Fibre Channel

FC Switch

FC SAN

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 33: Networking Technologies: Concepts in Internal and External

33

EDUCATION

Logical Units0 1 2 62 63

PeripheralController

Adapter

Target

Logical Units0 1 2 62 63

PeripheralControllerAdapter

Target

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Virtual SCSI Cables Fibre Channel

FC Switch

FC Point-to-Point: Direct Routing

FC Loop: Arbitrate, Open Loop Circuit

FC Fabric: Switch Routing

Blocks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 34: Networking Technologies: Concepts in Internal and External

34

EDUCATION

Virtual SCSI Cables SAS

Target

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Adapter

Target

Logical Units0 1 2 62 63

PeripheralController

Adapter

SAS

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 35: Networking Technologies: Concepts in Internal and External

35

EDUCATION

Virtual SCSI Cables SAS

Target

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Adapter

SAS Expander

Target

Logical Units0 1 2 62 63

PeripheralController

Adapter

SAS

Serial Attached SCSI: Open, Information, Close

Blocks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 36: Networking Technologies: Concepts in Internal and External

36

EDUCATION

Virtual SCSI Cables IP Storage (iSCSI)

Target

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Adapter

Target

Logical Units0 1 2 62 63

PeripheralController

Adapter

IP Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 37: Networking Technologies: Concepts in Internal and External

37

EDUCATION

Virtual SCSI Cables IP Storage (iSCSI)

Target

Logical Units0 1 2 62 63

PeripheralController

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

HostAdapter

HostComputer

Initiator

Adapter

Ethernet Switch

Target

Logical Units0 1 2 62 63

PeripheralController

Adapter

Blocks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 38: Networking Technologies: Concepts in Internal and External

38

EDUCATION

M C Escher – Ascending & Descending Monks

Page 39: Networking Technologies: Concepts in Internal and External

39

EDUCATION

SAS Storage Controller Perspective

HostHBA

Disks Root

Edge Expanders

Fanout Expander

SAS “Expanse”

Service Delivery Subsytem

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 40: Networking Technologies: Concepts in Internal and External

40

EDUCATION

SAS Storage Network Perspective

HBA Disk

Edge Expanders

Fanout Expander

Root

SAS “Expanse”Service Delivery Subsytem

End Devices Expanders

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 41: Networking Technologies: Concepts in Internal and External

41

EDUCATION

SAS Storage Network: Host – Disk Perspective

HBAs Disks

Edge Expanders

Fanout Expander Roots

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 42: Networking Technologies: Concepts in Internal and External

42

EDUCATION

SAS Storage Network: Host – Disk Perspective

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 43: Networking Technologies: Concepts in Internal and External

43

EDUCATION

Parts of a Physical Transport Network?

1. 2. 3.Media or the “appearance of”media

Ports - 2 or more (transceivers)

Protocol

?

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 44: Networking Technologies: Concepts in Internal and External

44

EDUCATION

Port vs. PHY = Ports vs. Docks

Fibre Channel, Ethernet

1 Port – 1 PHY

SAS, Infiniband

1 Port – Many PHYs

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 45: Networking Technologies: Concepts in Internal and External

45

EDUCATION

Striping the Transmission Word1 Link - 4 Phy Lanes

0 1 2 30 1 2 3

0 1 2 3

PCIe, InfiniBand does Striping (4x, 12x)SAS is not Striping – See connection path

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 46: Networking Technologies: Concepts in Internal and External

46

EDUCATION

Striping the Transmission Word1 Link - 4 Phy Lanes

0 1 2 30 1 2 3

0 1 2 3

10 Gbps Fibre Channel, PCIe, InfiniBand does StripingSAS is not Striping – See connection path

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 47: Networking Technologies: Concepts in Internal and External

47

EDUCATION

SAS Expander – Open, Arbitration, Open Accept

Node Node

Node NodeExpander

ARB 02, 03

Frame Primitive

ID03

ID04

ID01

ID02OPEN Accept

OPEN 02, 03

AIPAIP Done

OPEN 02, 03

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 48: Networking Technologies: Concepts in Internal and External

48

EDUCATION

SAS Expander – Move

Node Node

Node NodeExpander

Frame Primitive

ID03

ID04

ID01

ID02

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 49: Networking Technologies: Concepts in Internal and External

49

EDUCATION

SAS Expander – Move

Node Node

Node Node

ID03

ID04

ID01

ID02

Expander

Frame Primitive

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 50: Networking Technologies: Concepts in Internal and External

50

EDUCATION

What is a Physical Transport Network?

• A group of connected systems - at least two!– Host, Node, Computer, Server, Workstation, Printer, Router, Storage Device

Network Interface Cards (NICs), Host Bus Adapters (HBAs), Storage Adapters (SAs)

• A media or the “appearance of” media– Unshielded Twisted Pair(s) (UTP), Coaxial Cable(s), Multi-Mode Fiber(s),

Leased Line(s), RF• A connection protocol or method

– Ethernet, Token Ring, PPP, Frame Relay, ATM, Fibre Channel, SONET/SDH, DWDM

• Switches create an appearance of media

Router

FileServer

Switch

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 51: Networking Technologies: Concepts in Internal and External

51

EDUCATION

Routing in Switches

• Routing

• Switch frame “steering” over one “appearance of media”

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 52: Networking Technologies: Concepts in Internal and External

52

EDUCATION

Routing in Routers

• Routing

• Router packet “forwarding”across many “appearances of media”– A Day in the Life at Manhattan starts at

home: – Walk to the bus stop– Get on the bus– Take the bus to the dock – Get off the bus and walk on to the Ferry boat – Sail across the river– Get off the boat and hop in a cab.– Drive to the office and get out of the cab.– Run to the elevator and hop in– Jump off the elevator and walk to your desk

– How many subnets? 9Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 53: Networking Technologies: Concepts in Internal and External

53

EDUCATION

An Internetwork: Switch vs. Router

• Routers join appearances of media

FileServer

Router

Switch

FileServer

Router

Switch

FileServer

Switch

FileServer

Switch

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 54: Networking Technologies: Concepts in Internal and External

54

EDUCATION

Router vs. Gateway

FileServer

Router

Switch

FileServer

Router

Switch

FileServer

Switch

FileServer

SwitchGateway

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 55: Networking Technologies: Concepts in Internal and External

55

EDUCATION

Circuit Switch – Packet Switch

• Fixed path whether moving or not versus routing frames or packets on the current best path as they arrive and leave

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 56: Networking Technologies: Concepts in Internal and External

56

EDUCATION

Circuit Switch

• SAS Expanders Frame

• FC Class 1 Frame

• POTS (Plain Old Telephone Network)

• Circuit Construction

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 57: Networking Technologies: Concepts in Internal and External

57

EDUCATION

Packet (Frame) Switch

• FC Class 3 Frame (switch)

• IP Packet (router)

• VOIP

• Frame Direction

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 58: Networking Technologies: Concepts in Internal and External

58

EDUCATION

Connection-Oriented vs. Connectionless

• Not path related! • Connection-oriented requires a state exchange

to track communications• Connectionless takes things as they come

Telephone TelegraphNetworking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 59: Networking Technologies: Concepts in Internal and External

59

EDUCATION

• Internet Protocol Suite– TCP connections start with negotiation setup between hosts

• Similar to telephone connection– Call, Answer, Talk, Hang up

• UDP connectionless– IP Connectionless – Handles packets as they come

• Fibre Channel Class 2, 3 Switch Routing– Connectionless – Handles frames as they come

TCP

TCPIP

IPIP

IP

Connection-Oriented vs. Connectionless

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 60: Networking Technologies: Concepts in Internal and External

60

EDUCATION

Connection-Oriented FC Sessions - Login

Fabric

N-Port Login

FC-4 Process

Login

Fabric Login

Fabric Login

LinkActive

LinkActive

LinkActive

Server Storage

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 61: Networking Technologies: Concepts in Internal and External

61

EDUCATION

Layered Protocols

• Protocol Data Units (PDUs)

• They are both cake – Different ways to provide similar function!

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 62: Networking Technologies: Concepts in Internal and External

62

EDUCATION

OSI & IPS (TCP/IP)

Transport

Session

Presentation

Application

Physical

Data Link

Network

End User Data

Transport

ApplicationServices

NetworkInterfaceSublayer

Internet

End User DataOpen Systems Interconnection Internet Protocol Suite

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 63: Networking Technologies: Concepts in Internal and External

63

EDUCATION

OSI & Fibre Channel Architecture

Transport

Session

Presentation

Application

Physical

Data Link

Network

End User Data

FC0 - Physical

FC2 - Framing/Flow ControlFC1 - Encode/Decode

FC3 - Common Services

FC4 - Upper Layer Mapping

Channels

IPI, SCSI,HIPPI, SBCCS

Networks

802.2 (LE),IP, ATM

End User Data

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 64: Networking Technologies: Concepts in Internal and External

64

EDUCATION

OSI & Fibre Channel Architecture

Transport

Session

Presentation

Application

Physical

Data Link

Network

End User Data

FC0 - PhysicalFC1 - Encode/Decode

FC3 - Common Services

FC4 - Upper Layer Mapping

Channels

IPI, SCSI,HIPPI, SBCCS

Networks

802.2 (LE),IP, ATM

End User Data

FC2 - Framing/Flow Control

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 65: Networking Technologies: Concepts in Internal and External

65

EDUCATION

Fibre Channel & IPS

Internet Protocol SuiteFibre Channel

Transport

ApplicationServices

NetworkInterfaceSublayer

Internet

FC0 - Physical

FC2 - Framing/Flow ControlFC1 - Encode/Decode

FC3 - Common Services

FC4 - Upper Layer Mapping

Channels

IPI, SCSI,HIPPI, SBCCS

Networks

802.2 (LE),IP, ATM

FC2 - Framing/Flow Control

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 66: Networking Technologies: Concepts in Internal and External

66

EDUCATION

Flow Control

• Fibre Channel– Credit-Based Fibre Channel– Advertise Credit

• Internet Protocol Suite– TCP Sliding Window– IP Datagram Drop Congestion Control– Advertise Larger Windows

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 67: Networking Technologies: Concepts in Internal and External

67

EDUCATION

SAS Flow Control-

Expanders – Circuit Switches

HBA Switch SASwitch

Frame = ~1K Max

Credit R-RDY = 1 Dword

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 68: Networking Technologies: Concepts in Internal and External

68

EDUCATION

Fibre Channel Buffer-to-Buffer (Class 2, 3)

N-Port E-Port

HBA SwitchN-Port

SA

F-Port

SwitchF-Port

Frame = ~2K Max

Credit R-RDY = 1 Word

B-B B-B B-B

E-Port

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 69: Networking Technologies: Concepts in Internal and External

69

EDUCATION

Fibre Channel End-to-End (Class 1, 2)

Fabric

N-PortE-Ports

N-PortF-PortF-Port

Frame

ACK Frame

Node Node NodeNode

E-E

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 70: Networking Technologies: Concepts in Internal and External

70

EDUCATION

Internet - TCP Sliding Windows

• Sliding Windows– Reduces overhead and delay of positive acknowledgments– Allows sending of multiple segments before ACK is required

• Window Advertisements– Receivers advertise window sizes– Usually MSS multiple– Maximum 65,535 Bytes– Prevent “Silly window” syndrome

• Slow Start– Congestion Window– Increments till congestion detected

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 71: Networking Technologies: Concepts in Internal and External

71

EDUCATION

TCP Window Scale Factor

• Multiplier for window field• Window field only 16 bits for maximum of 65,535

bytes• Both systems in a connection must agree to use

this• Allows a maximum of 16 million bytes

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 72: Networking Technologies: Concepts in Internal and External

72

EDUCATION

TCP Sliding Windows – TCP Segments

01 02 03 04 05 06 07 08 09 10

01 02 03 04 05 06 07 08 09 10

Send 6 Segments without Acknowledgement

2 Acknowledgements

Send 2 More Segments

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 73: Networking Technologies: Concepts in Internal and External

73

EDUCATION

TCP Sliding Windows

11 12 13 14 15 16 17 18 19 20

Increasing

11 12 13 14 15 16 17 18 19 20

Decreasing

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 74: Networking Technologies: Concepts in Internal and External

74

EDUCATION

Payload Integrity

• Do you always want payload integrity?

• Not necessarily with Video and Voice

• Performance vs. Quality tradeoffs– A pixel in a online-rented

movie frame – A medical imaging application

just prior to heart surgery where you want to be sure the speck of grey isn’t line noise!

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 75: Networking Technologies: Concepts in Internal and External

75

EDUCATION

Names & Addresses

Name

Address

Route

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 76: Networking Technologies: Concepts in Internal and External

76

EDUCATION

Name, Address & Route

• Addressing and Identification schemes

• Address is used to locate

• Names or identifiers uniquely represent something or someone

Name

Address

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 77: Networking Technologies: Concepts in Internal and External

77

EDUCATION

Name, Address & Route• Name is a specific identification

– Machine, User, Application - Unique target for data• Internet

– Fully Qualified Domain Names WWW.SNIA.ORG – Media Access Control MAC ID – Physical Port Identifier

• Storage – World Wide Node Names WWNN (Fibre Channel)– World Wide Port Names WWPN (Fibre Channel)– Enterprise Unique Identifier EUI (iSCSI)– iSCSI Qualified Name IQN (iSCSI)

• Address identifies where the target is located– Hardware, Network, Port, Socket

– Port ID FCID (Fibre Channel)– IP Address

• Route identifies path to get to the destination

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 78: Networking Technologies: Concepts in Internal and External

78

EDUCATION

Transport Addressing Ports

• Ports are used to deliver information to relevant application services

• Well-known and “Not-so-well-known” Transient

• Comparable to Well known FCID Port ID Addresses– FFFFFC Name Server– FFFFFD Fabric Controller– FFFFFE Fabric Port Login Server

Port Name Description 20 FTP-3DATA File Transfer (Data Channel) 21 FTP File Transfer (Control Channel) 22 TELNET Telnet 25 SMTP Simple Mail Transfer 53 DOMAIN Domain Name Service DNS Domain Name Services

3260 ISCSI Internet SCSI

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 79: Networking Technologies: Concepts in Internal and External

79

EDUCATION

Comparative Address Ranges• 1 Address = 1 Gram• SATA Point-to-Point

– 20 = 1

• SATA II “Port Multiplier”– 24 = 16

• SAS “Expanders”– 214 = 128 x 128 = 16,384

• All Fibre Channel – 224 = 16,777,216

• All IPv4 – 232 = 4,294,967,296

• All IPv6 – 2128 = 340,282,366,920,938,463,463,374,607,431,768,211,456

X 56 Billion

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 80: Networking Technologies: Concepts in Internal and External

80

EDUCATION

Hierarchical, Routable Addressing

• IP addresses can be divided into three parts– Network Address (NETID)– Subnetwork Address using (Subnetwork Mask 255.255.255.0)– Host IP Address

• Example 172.16.2.6– Network 172.16.0.0, Subnetwork 172.16.2.0, Host IP Address

172.16.2.6

• FC addresses can be divided into three parts– Domain (Domain ID)– Area (Area ID) – Device (Device ID)

• Example 0102EF– Domain 01 (Switch 01), Area 02 (Port 02 on the Switch), Device EF

(ALPA of Loop Device)

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 81: Networking Technologies: Concepts in Internal and External

81

EDUCATION

Address Assignment• DHCP Mechanism for allocating and assigning IP

addresses to clients– Automatic: Host requesting an address are provided with a

permanent IP address– Dynamic: Hosts requesting an address are provided with a

temporary address– Manual: Host IP addresses are manually configured and DHCP

just delivers these assignments

• Fibre Channel– Loop Initialization LISM, LIFA, LIPA, LIHA, LISA, LIRP, LILP,

Close• Select Master, Fabric Assigned, Previously

Assigned, Hard Assigned, Soft Assigned, Report Position, List Position, Close

– Fabric Login Controller– Point-to-Point - Lowest World Wide Port Name (WWPN) Assigns

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 82: Networking Technologies: Concepts in Internal and External

82

EDUCATION

Name Service

• Internet– Domain Name Services for IP

• Fully Qualified Domain Name FQDN to IP Address• Running on distributed servers

• Storage– Name Services for Fibre Channel

• WWPN to Fibre Channel Identifier FCID, Fabric service running in the Switch

– iSNS Storage Name Service (iSNS) for iSCSI

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 83: Networking Technologies: Concepts in Internal and External

83

EDUCATION

Segmentation & Offload

• Segmentation, Convergence and Reassembly

• Offload for Performance Acceleration

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 84: Networking Technologies: Concepts in Internal and External

84

EDUCATION

Traditional FC SAN

Host Applications

Host Applications

File ManagerFile ManagerDevice DriverDevice DriverSCSI ProtocolSCSI Protocol

Operating SystemI/O Interface

HBA

FC2FC1

FC3

FC4

FC0

Transport

ApplicationServices

Internet

Internet Protocol Suite

Host NIC

NetworkInterfaceSublayer

Ethernet

FC2

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 85: Networking Technologies: Concepts in Internal and External

85

EDUCATION

Read 8K (6 FC frames with 4 - 2K Frames)

Host

HBA

FC2FC1

FC3

FC4

FC0

FC2

Fibre Channel Multi-Frame Sequence Minimizes Server I/O interrupts - 2

Read CommandDataDataDataDataStatus

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 86: Networking Technologies: Concepts in Internal and External

86

EDUCATION

Traditional NAS

Host Applications

Host Applications

File ManagerFile ManagerDevice DriverDevice DriverSCSI ProtocolSCSI Protocol

Operating SystemI/O Interface

HBA

FC2FC1

FC3

FC4

FC0

Transport

ApplicationServices

Internet

Internet Protocol Suite

Host NIC

NetworkInterfaceSublayer

Ethernet

NFS/CIFS

FC2

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 87: Networking Technologies: Concepts in Internal and External

87

EDUCATION

Get 8K (7 Ethernet Frames with 6 - 1.5K Frames)

Host

Host IP Fragmentation - 6 IP Packet Fragments, 6 – 1500 byte MTU Ethernet Frames, Maximizes Server I/O interrupts - 7

NIC

NetworkInterfaceSublayer

Ethernet

Transport

ApplicationServices

Internet

Get FileDataDataDataDataDataData

NFS / CIFS / iSCSI

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 88: Networking Technologies: Concepts in Internal and External

88

EDUCATION

iSCSI Full Offload

Host Applications

Host Applications

File ManagerFile ManagerDevice DriverDevice DriverSCSI ProtocolSCSI Protocol

Operating SystemI/O Interface

Transport

ApplicationServices

Internet

Internet Protocol Suite

Host IntelligentNIC/HBA

NetworkInterfaceSublayer

GigabitEthernet

iSCSI

TCP

IP

iSCSI

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 89: Networking Technologies: Concepts in Internal and External

89

EDUCATION

Read 8K (8 Ethernet frames with 6 – 1.5K)

Host Offload IP Fragmentation (Like FC Multi-Frame Sequence)Minimizes Server I/O interrupts - 2

Read CommandDataDataData

Status

IntelligentNIC/HBA

NetworkInterfaceSublayer

GigabitEthernet

TCP

IP

iSCSI

DataData

Data

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 90: Networking Technologies: Concepts in Internal and External

90

EDUCATION

Serial Scaling

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 91: Networking Technologies: Concepts in Internal and External

91

EDUCATION

Serial Scaling

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 92: Networking Technologies: Concepts in Internal and External

92

EDUCATION

Serial Link Bit Rate Scaling

40 Bits = Tword

1 Bit

1 Bit @ 1.5 Gbps = .666ns

Transmit Receive

1 Bit @ 3.0 Gbps = .333 ns

1 Bit40 Bits = Tword

Assume 40 Folds

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 93: Networking Technologies: Concepts in Internal and External

93

EDUCATION

Bandwidth

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 94: Networking Technologies: Concepts in Internal and External

94

EDUCATION

iSCSI Overhead

iSCSIStandard Ethernet

Ethernet with Non-Standard Jumbo Frames

iSCSI/TCP/IP/ Standard Ethernet Overhead

iSCSI/ TCP/IP/ Ethernet Overhead with Non-Standard Ethernet Jumbo Frames

iSCSI Data Out or Data in PDU Header Bytes 52 52

TCP Header Bytes 20 20

IPv4 Header Bytes 20 20

Preamble Bytes 7 7 7 7SFD Bytes 1 1 1 1

Destination Address Bytes 6 6 6 6Source Address Bytes 6 6 6 6Length/Type Bytes 2 2 2 2

Maximum Data/Pad Bytes 1,500 9,000 1,500 9,000FCS Bytes 4 4 4 4

Ethernet Frame overhead including Preamble and SFD fields Bytes 26 26 26 26

Total Overhead Bytes 52 52 144 144

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 95: Networking Technologies: Concepts in Internal and External

95

EDUCATION

iSCSI Performance

iSCSIStandard

Ethernet

Ethernet with Non-Standard Jumbo Frames

iSCSI/TCP/IP/ Standard Ethernet Overhead

iSCSI/ TCP/IP/ Ethernet Overhead with Non-Standard Ethernet Jumbo Frames

Total Overhead Bytes 52 52 144 144

Payload Bytes 1,500 9,000 1,500 9,000

Efficiency % 98.30% 99.43% 91.24% 98.43%

Bit Rate gbps 1.25 1.25 1.25 1.25

Data Rate MBps 122.87 124.28 114.05 123.03

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 96: Networking Technologies: Concepts in Internal and External

96

EDUCATION

Fibre Channel Performance

Bytes Bytes Bytes

Fibre Channel

Fibre Channel Class 3

SCSI FC DATA Out or IN / Fibre Channel Class 3

SCSI FC DATA Out or IN / Fibre Channel Class 2

SOF Bytes 4 4 4 4 4 4 4 4 4Frame Header Bytes 24 24 24 24 24 24 24 24 24Maximim Payload Bytes 2,112 2,048 2,048 2,112 2,048 2,048 2,112 2,048 2,048CRC Bytes 4 4 4 4 4 4 4 4 4EOF Bytes 4 4 4 4 4 4 4 4 4InterFrame Word Minimum Bytes Bytes 24 24 24 24 24 24 24 24 24Acknowledgement Frame Overhead 60 60 60Total Overhead Bytes 60 60 120 60 60 120 60 60 120

Payload Bytes 2,112 2,048 2,048 2,112 2,048 2,048 2,112 2,048 2,048

Efficiency % 97.24% 97.15% 91.92% 97.24% 97.15% 91.92% 97.24% 97.15% 91.92%

Bit Rate gbps 1.0625 1.0625 1.0625 2.125 2.125 2.125 4.25 4.25 4.25

Data Rate MBps 103.31 103.23 97.67 206.63 206.45 195.33 413.26 412.90 390.66

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 97: Networking Technologies: Concepts in Internal and External

97

EDUCATION

SAS PerformanceBytes Bytes

Serial Attached SCSI SAS SAS

SOF Bytes 4 4

Frame Header Bytes 24 24

Maximum Payload Bytes 1,024 1,024

CRC Bytes 4 4

EOF Bytes 4 4

InterFrame Word Minimum Bytes (1 Align every 2048 dwords) Bytes 2 2

R-Rdy Frame Overhead Bytes 4 4

Acknowledgement Frame Overhead Bytes 4 4

Total Overhead Bytes 46 46

Payload Bytes 1,024 1,024

Efficiency % 95.70% 95.70%

Bit Rate gbps 1.5 3

Data Rate MBps 143.55 287.10

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 98: Networking Technologies: Concepts in Internal and External

98

EDUCATION

Bandwidth does not equal bit rate

Bit Rate Efficiency Data Rate Full DuplexStandard Gigabit

Ethernet1.25 Gbps 98.3% 123 MBps 246 MBps

iSCSI / TCP / IP / Gigabit Ethernet Standard MTU

1508

1.25 Gbps 91.2% 114 MBps 228 MBps

iSCSI / TCP / IP / Gigabit Ethernet Jumbo Frames

MTU 9000

1.25 Gbps 99.43% 124.28 MBps 248.56 MBps

Standard SCSI FCP Fibre

Channel Class 3

1.0625 Gbps 97.2% 103.5 MBps 207 MBps

SAS 1.5 Gbps 95.7% 143.55 MBps 287.10 MBps

SAN Performance ComparisonSAN Performance Comparison

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 99: Networking Technologies: Concepts in Internal and External

99

EDUCATION

Harry Newton’s Telecom Dictionary 17th

edition

• Delay– The wait time between two events, such as the time from when a

signal is sent to the time it is received.– There are all sorts of reasons for delays

• Propagation Delay• Queuing Delay• Processing Delay• Rotational Delay• Satellite Delay

• Latency– A fancy term for waiting delay. The time it takes to get

information through a network.– Accumulated delay

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 100: Networking Technologies: Concepts in Internal and External

100

EDUCATION

Delay & Latency• Delay

– A metric that measures how long something is not happening

– (A packet waiting in a queue)• Latency

– A metric that measures how long it is between things happening

– (The time between when a packet arrives at a device and leaves a device

• Throughput– Delay and Latency affect

throughput

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 101: Networking Technologies: Concepts in Internal and External

101

EDUCATION

Throughput does not equal Bandwidth

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 102: Networking Technologies: Concepts in Internal and External

102

EDUCATION

Network Components

• Network Topologies– Data Networks

• Point-to-Point, Bus, Token Ring, Fabric– Storage Networks

• Bus, Arbitrated Loop, Point-to-Point, Fabric– Hybrids

• Network Types– Local Area, Metropolitan Area, Wide Area, Storage Area

• Backbone Networks• Network Interconnectivity

– Repeaters, Hubs, Bridges, Switches, Routers, Gateways

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 103: Networking Technologies: Concepts in Internal and External

103

EDUCATION

Storage Network ChoicesIntegrated Device Electronics (IDE)Small Computer Systems Interface (SCSI) Bus Enterprise System Connectivity (ESCON)Serial ATA, Serial Attached SCSI

Fibre Channel

IP Storage

InfiniBand

iSCSI / TCP / IP / GE

“Very Small “Very Skinny

Fibre Channel

Storage over IPiSCSI / TCP / IP / GE

internet SCSI (iSCSI)Transmission Control Protocol (TCP)Internet Protocol (IP)Gigabit Ethernet (GE)

internet Fibre Channel Protocol (iFCP) Fibre Channel Internet Protocol (FCIP)internet Storage Name Service (iSNS)

InfinibandSRPiSCSI Extension over RDMA (ISER)

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 104: Networking Technologies: Concepts in Internal and External

104

EDUCATION

Semantic “Anal”yst: Beware of Semantics

Switch

Hub

Router

GatewayNetwork terminology misuse is rampant!

• By the Storage Network Industry

• By Vendors

• By Media

• By Educators

Confusing the customer!

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 105: Networking Technologies: Concepts in Internal and External

105

EDUCATION

Beware of Semantics

Switch

BROUTERSWOUTER

GROUTER

Hub

Router

Gateway

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 106: Networking Technologies: Concepts in Internal and External

106

EDUCATION

Beware of Semantics

Director BridgeSwitch

BROUTERSWOUTER

GROUTER

HubRepeater Loop Switch

Router

Gateway

Smart

Intelligent

Enterprise

StorageCluster

Virtual

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 107: Networking Technologies: Concepts in Internal and External

107

EDUCATION

Plumbing is Beautiful

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 108: Networking Technologies: Concepts in Internal and External

108

EDUCATION

Storage Networking Futures

• Upper layer protocol changes– Lazy and Hyper reads and writes– System SAN awareness, Dynamic discovery

• Higher bandwidth rates for aggregation and higher speed processors for servers, storage and adapters

• Focus on PCI Express and Infiniband• External SANs with Fibre Channel and

iSCSI/TCP/IP/GE• Internal SANs with SAS and SATA• NAS & SAN integration

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 109: Networking Technologies: Concepts in Internal and External

109

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage

• Many – Storage Network Solutions– Storage Network Technologies– Many Opportunities for Benefit

and Value

• Thanks for coming!– Hope this helps!!– Please fill out your evaluation

forms!!!– Good luck with Storage Networks

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 110: Networking Technologies: Concepts in Internal and External

110

EDUCATION

Outline

NetworkingTechnologies

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Page 111: Networking Technologies: Concepts in Internal and External

111

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Q&A / Feedback

• Please send any questions or comments on this presentation to the SNIA:

[email protected]

Many thanks to the following individuals for their contributions to this tutorial.

SNIA Education Committee

Howard Goldstein – Tutorial Manager, Developer & Presenter

Greg Schulz – Tutorial ReviewSW Worth – Tutorial Review

Page 112: Networking Technologies: Concepts in Internal and External

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage©Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

HGAI Storage Networking Web Links & Bibliography

For more info contact:[email protected]

Page 113: Networking Technologies: Concepts in Internal and External

113

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Web Links

http://www.t10.org T10 (SCSI)http://www.ncits.org/ The NCITS home pagehttp://www.symbios.com/t10.1/ The (former) T10.1 Task Group

(SSA) home pagehttp://www.t11.org/ The T11 Technical Committee

home page (Fibre Channel, HIPPI, IPI, SBCON)

http://www.scsita.org The T13 Technical Committee home page (ATA & ATAPI)

http://www.fibrechannel.com Fibre Channel Industry Association (a trade association)

Page 114: Networking Technologies: Concepts in Internal and External

114

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Web Links

http://www.infostor.com Storage Industry informationhttp://www.snia.org Storage Networking Industry

Association (a trade association)

http://www.1394ta.org/ IEEE 1394 Trade Associationhttp://www.infinibandta.org Infiniband Trade Associationhttp://www.ultra160-SCSI.com/ Information about Ultra160 SCSIhttp://searchstorage.com Useful storage informatonhttp://www.jobstor.com Jobs in the storage industryhttp://www.asnp.org Association of Storage Network

Professionalshttp://www.storagenetworking.org Storage Networking User

Groups

Page 115: Networking Technologies: Concepts in Internal and External

115

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Fibre Channel – A Comprehensive Introduction Robert W. Kembel• IP SANs– A Guide to iSCSI, iFCP, and FCIP Protocols for Storage Area

Networks Tom Clark• Building Storage Networks Marc Farley• iSCSI – The Universal Storage Connection John L. Hufferd• Resilient Storage Networks – Designing Flexible Scalable Data

Infrastructures Greg Schulz• Using SANs and NAS W. Curtis Preston• Storage Area Network Essentials – A Complete Guide to

Understanding and Implementing SANs Richard Barker & Paul Massiglia

These texts were used in creating this course. Some of the course material contains references from this material

Page 116: Networking Technologies: Concepts in Internal and External

116

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Storage Security – Protecting SANs, NAS, and DAS John Chirillo & Scott Blaul

• Inside Windows Storage – Server Storage Technologies for Windows 2000, Windows 2003, and Beyond Dilip C. Naik

• Successfully Implementing and Scaling SANs in the Enterprise - A Planning Guide for Storage Networking Professionals SearchStorage.Com

• The Holy Grail of Network Storage Management Jon William Toigo• Storage Areas Networks for Dummies Christopher Poelker & Alex Nikitin• Practical Storage Area Networking Daniel Pollack• Designing Storage Area Networks – A Practical Reference for

Implementing Fibre Channel and IP SANs Tom Clark

These texts were used in creating this course. Some of the course material contains references from this material

Page 117: Networking Technologies: Concepts in Internal and External

117

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Storage Area Networks – Designing and Implementing a Mass Storage System Ralph H. Thornburgh & Barry J. Schoenborn

• Cisco Storage Networking Architecture Poster Cisco Press• IP Storage Networking – Straight to the Core Gary Orenstein• Fibre Channel for Mass Storage Ralph H. Thornburgh• Making SCSI Work – A Practical Guide The Paralan Staff• Fibre Channel for SANs Alan F. Benner• The Book of SCSI – A Guide for Adventurers Peter M. Ridge• The Holy Grail of Data Management – Modeling, Evaluating,

Comparing, Implementing, Storage Management Options, and Solutions Jon William Toigo

• Storage Network Performance Analysis Huseyin Simitci

These texts were used in creating this course. Some of the course material contains references from this material

Page 118: Networking Technologies: Concepts in Internal and External

118

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Fibre Channel – Gigabit Communications and I/O for Computer Networks Alan F. Benner

• Building SANs with Brocade Fabric Switches – How to Design, Implement, and Maintain Storage Area Networks (SANs) with Brocade Fabric Switches Chris Beauchamp, Josh Judd, & Benjamin Kuo

• Multiprotocol Routing for SANs Josh Judd• The Fibre Channel Bench Reference Jeffrey D. Stai• The SCSI Bus & IDE Interface – Protocols, Applications & Programming

Friedhelm Schmidt• Fibre Channel Switched Fabric Robert W. Kembel• Fibre Channel Arbitrated Loop Robert W. Kembel• Storage Networks Daniel Worden

These texts were used in creating this course. Some of the course material contains references from this material

Page 119: Networking Technologies: Concepts in Internal and External

119

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Disaster Recovery Planning – Preparing for the Unthinkable Jon William Toigo

• Fibre Channel Volume 1: The Basics Gary R. Stephens & Jan V. Dedek• Small Computer System Interface – An In-Depth Exploration Of ….

David Deming• What is Fibre Channel? Ancot Corporation• Fibre Channel Connection to the Future Fibre Channel Association FCA• Introduction to Storage Area Network, SAN Ravi Kumar Khattar, Mark S.

Murphey, Giulio John Tarella, Kjell E. Nystrom

These texts were used in creating this course. Some of the course material contains references from this material

Page 120: Networking Technologies: Concepts in Internal and External

120

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• FireWire System Architecture Second Edition IEEE 1394a MindShare, Inc. Don Anderson

• PCI-X System Architecture MindShare, Inc. Tom Shanley• Storage Virtualization Technologies for Simplifying Data Storage and

Management Tom Clark• Serial Attached SCSI: Beyond the Basics SAS Knowledge Tek Author:

Hugh Curley• SAS Storage Architecture – Serial Atached SCSI MindShare, Inc. Mike

Jackson

These texts were used in creating this course. Some of the course material contains references from this material

Page 121: Networking Technologies: Concepts in Internal and External

121

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Securing Storage A Practical Guide to SAN and NAS Security Himanshu Dwivedi

• Storage Network Security – SNIA Technical Tutorial Roger Cummings & Hugo Fruehauf

• Storage Virtualization – SNIA Technical Tutorial Frank Bunn, Nik Simpson, Robert Peglar & Gene Nagle

• Storage Network Management – SNIA Technical Tutorial Roger Cummings

• Network Storage Terms & Acronyms Storage Networking Industry Association SNIA

These texts were used in creating this course. Some of the course material contains references from this material

Page 122: Networking Technologies: Concepts in Internal and External

122

EDUCATION

Networking Technologies: Concepts in Internal and External Networked Storage© Copyright 2007 Howard Goldstein Associates, Inc. HGAI All Rights Reserved.

Bibliography

• Ethernet The Definitive Guide Charles E. Spurgeom• Gigabit Ethernet for Metro Area Networks Paul Begell• Gigabit Ethernet Networking David G. Cunningham, Ph.D. and William G.

Lane, Ph. D.• High Performance TCP/IP Networking – Concepts, Issues, and

Solutions Mahbub Hassan, Raj Jain• Gigabit Networks – Standards and Schemes for Next-Generation

Networking Paul Izzo

These texts were used in creating this course. Some of the course material contains references from this material