storage area networks the basics. storage area networks sans are designed to give you: more disk...

Storage Area Networks

The Basics

Storage Area Networks

SANS are designed to give you:• More disk space• Multiple server access to a single disk

pool• Better performance• Option of disk distributed across

multiple locations

Direct Attached Storage

Classically, for storage we had a single box with a bunch of disks attached:

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

SCSI Bus

LUN0 LUN1 LUN2

Server

PublicNetwork

Attached Storage

The server speaks to the SCSI disks using a command language:

• Read from LUN0, Block 123• Write to LUN1, Block 456All this goes over the SCSI bus, which is directly

attached to the server; only that server has access to the bus

The server would create a filesystem on the disk(s) and could then make the disk available to other computers via NFS, Samba, etc.

Network Attached Storage

This idea is easily extended to an appliance approach. Configure a utility box with some disk that does only NFS or Samba/SMB, place on network


are needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

SCSI Bus

Public Network



NFS ClientNFS Server

NAS Server

NAS and Servers


are needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

SCSI Bus

Public NetworkQuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.

NFS Server

NAS Server

Redundant web servers share the same data--but they both talk to the same NFS server

Web server, dataNFS mounted



Web server, dataNFS mounted

Attached Storage

We can also do things like place a RAID array on the NAS server.

This works, but it has some limitations:• If the server goes down, there is no access to the

disk• File sharing goes through the network storage

server and across the network, which can be slow• Limitations on location of disks--must be near

server, within range of the disk bus• Adding or subtracting disk space can be difficultWhat we want is a shared disk pool that all servers

can access

Storage Area Network

What we want is something that looks like this:


are needed to see this picture.QuickTime™ and a








are needed to see this picture.NFS ClientPublic Ethernet Net

SAN Participants

Disk Pool


Notice:• You can take down a server and still maintain access to the

disk pool via the other SAN participants• Disk added to the pool is available to all servers, not just one• Shared, high speed access to the disk pool; can run clustered

copies of SQL database or web server if the SQL databases or web servers are also SAN participants

• Can still serve up the disk pool via an NFS or SMB server on a SAN-connected box

• “serverless backups”--just send command to copy blocks from disk A to disk B. Snapshots easier, shortened backup windows--you can have a SAN particpant handle moving a volume to tape


So how does this work? It’s a scaled up version of the old system.

• The commands being sent are the same disk standard commands: either SCSI or ATA disk bus commands, READ, WRITE, etc.

• The network connecting the SAN servers to the disk is typically (but not always) higher speed, eg FibreChannel

• Some extra glue to allow for concurrent access by more than one server--need a shared filesystem

• Special filesystems to allow for concurrent access


A popular choice:• SCSI for the bus commands (commands

sent over the wire)• Fiber Channel for the SAN network• EMC or similar for the glue volume

softwareFiber Channel is 2+ Gbit/sec, and can be

deployed across up to a 500m distance (sometimes) and up to 70 KM with special equipment


Another option is to use gigabit ethernet for the SAN networking.

• Cheap! Commodity equipment, don’t need to learn new Fiber Channel stuff, reuse existing gear

• But also lower performance--fibre channel has higher BW, and can use more of it.

ATA Over Ethernet

AoE uses ethernet plus ATA bus commands rather than SCSI. Low cost; but since ethernet frames are not routable all devices must be on the same network

iSCSI

iSCSI uses SCSI bus commands over ethernet, encapsulated inside of TCP/IP

• Cheap hardware!• Well supported in Linux, Solaris, and Windows

world• Because the SCSI is inside of TCP/IP, it is routable--

which means you can do a SAN across wide area networks (with lower performance due to latency) and do things like mirror for disaster backup, or across campus on high performance networks

• Processing TCP/IP takes some overhead; some use TCP offload chips

iSCSI

Each “disk”/LUN is a RAID array that understands iSCSI.











iSCSI

The green network is a dedicated (usually) gigabit ethernet network that carries the SCSI commands encapsulated inside TCP/IP. The red network connects the SAN participants to other clients not on the SAN

Important point: TCP/IP is routable. That means that (modulo latency) the devices can be located anywhere. We could have a iSCSI SAN participant in Root Hall, and one in Spanagel. The Root iSCSI server can access the disk pool in Spanagel

We could also have a volume located at Fleet Numeric in the same SAN

The price we pay for this is having to process the TCP/IP overhead as iSCSI commands go up the network protocol stack. This can be alleviated in part by TCP offload chips

Volume Software

Remember, the iSCSI targets are just block devices. iSCSI says nothing about concurrent access or multiple hosts accessing the same devices

For that we need a SAN Filesystem. This deconflicts concurrent access by hosts to the block devices

Volume Software











Vol1Vol2

SAN Software

The “volume software” allows you to build a concurrent access filesystem out of one or more LUNs

iSCSI

Example: Five compute servers need read access to one weather data set. If the servers are all on the SAN, they can directly access the data

Example: backup. Copy disk blocks directly, then have a tape drive SAN participant copy to tape

Example: storage expansion. Just add more disk, and it is available to all SAN participants

Competitors

iSCSI’s competitor is for the most part fibre channel. The concept of fiber channel is almost identical, but the SCSI commands are simply encapsulated in a fibre channel frame

Fibre channel is typically higher performance--more data can be pushed across FC, and there is much less overhead processing FC frames

BUT it is higher costATA Over Ethernet is very similar to FC in

concept--directly inserting the ATA commands in ethernet frames. But it seems to have less market penetration

storage area networks the basics. storage area networks sans are designed to give you: more disk...

Documents

network storage server

nfs server web server

smb server

near server

disk available

shared disk pool

disk standard commands

data nfs mounted web