august network announcement v3

© 2012 IBM Corporation

IBM Flex System Fabric SI4093 System Interconnect Module

Heather Richardson - Product MarketingHector Sanchez – Product Manager

Aug 2013

© 2012 IBM Corporation2 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.

This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.

Agenda

IBM Flex System Fabric SI4093 Module EN4093R Update IBM Flex System EN6131 40Gb Switch

Infrastructure ComponentsBeyond Blades



Data Center Challenges

Complexity Too many layers to manage Virtual or logical management

Multiple Management points Physical and Virtual Management Manage Server, Storage and I/O as one entity

Increase infrastructure utilization IT budgets are flat or decreasing Virtualization / Cloud and new applications demand more throughput and

performance Manage risk and plan for next decade

Reduce data center foot print – space, power and cost Focus on all aspects to increase hardware utilization Management is one of the biggest area of focus followed by hardware

spending and power



Intelligence is moving to the edge of the network

WANWAN

Layer2

SAN

Access Layer

Core Layer

Aggregation Layer

> 75%> 75%

< 2

5%<

25%

Servers

Network

Storage

Software

Intelligence is moving from the core to the edge– Opposite of what some of our competitors want!

Why this is happening:– Performance and latency demands

– Server and networking optimization is critical

– Convergence of LAN and SAN

– Virtualization of more east-west traffic

Customer Impact: – High Frequency Trading, Database intensive apps,

Messaging (MQ) and Highly Virtualized environments



NEW v 1.3 NEW v 1.3 NEW NEW

NEW POWER 7+NEW POWER 7+

NEWNEW

p270 p460

x222

p24L p260

The Flex System Portfolio Continues to Grow

EN4093RSI4093

Flex System Manager

Flex System V7000

Storwize V7000

x220 x240 x440 PCiE Expansion

Storage Expansion

EN6131 EN2092 FC5022 FC3171CN4093



Networking

IBM Flex System Fabric SI4093 System Interconnect ModuleN

etw

ork

ing

In

fras

tru

ctu

re

Simplifies chassis connectivity without

compromising performance◊

Suited for Clients looking for ‘unmanaged’ device in the

chassis◊

“Pay as you grow” scalability ◊

Seamless interoperability

• Designed for easy connectivity and management• Pre-configured from factory – enhances ‘out of box’

experience for Clients• Rapid network provisioning without configuration

errors• Investment Protection – scalable pay-as-you-grow

design reduces cost and complexity• Intra chassis switching for optimal performance

(Ex. vMotion)• Virtual Fabric capability reduces cost and improves

IO utilization• Seamless interoperability with other vendors

switches like Cisco, Juniper and Brocade

Base 10 x 10G

bE

SF

P+

#2 = 4x10G

bE

#1 = 2x40G

bE

1Gb

E M

gm

t

Total Ports

10Gb

to Server

10Gb

Uplinks

40Gb

Uplinks

Base System 14 10 0

Upgrade #1 28 10 2

Upgrade #2 42 14 2

Unmanaged 10Gb module with10/40Gb Uplinks

Unmanaged 10Gb module with10/40Gb Uplinks



Networking

IBM Flex System Fabric EN4093R with OpenFlow SupportSoftware Defined Networking

Net

wo

rkin

g I

nfr

astr

uct

ure

Increased IT efficiency with programmable networks

Increased IT efficiency with programmable networks

OpenFlow support enables Software Defined Networks

(SDN)◊

Programmable network enables easy customization

and innovation ◊

Create SDN-ready network architecture based on Open

Standards

• Reduce complexity by building the underlying network once

• Dynamic and rapid application deployment

• Automate network provisioning and interoperability

• Intelligent and dynamic multipath routing based on business policy

• Customize network architecture based on business policies and needs

• Centrally configure and enforce multi-tenant networks

• Available to new and existing Clients as feature update at no extra cost

• Reduce complexity by building the underlying network once

• Dynamic and rapid application deployment

• Automate network provisioning and interoperability

• Intelligent and dynamic multipath routing based on business policy

• Customize network architecture based on business policies and needs

• Centrally configure and enforce multi-tenant networks

• Available to new and existing Clients as feature update at no extra cost



Benefits Key Features

Reduced cost and complexity • Easy connectivity with preconfigured settings• Eliminates the need to configure each device or individual ports

thus reducing the number of management points• Lower TCO - Up to 42% less expensive than the competition*• Lower cabling and power costs

“Pay as you grow” scalability • Supports up to forty-two downlink 10GbE connections, fourteen uplink 10GbE SFP+ connections plus two 40GbE QSFP+ connections

• 2-3 X more bandwidth than the competition**

Simple Management • Default profile end host mode – plug and play just add cables• No Spanning Tree, loop free, port aggregation System Interconnect• End Host configuration viewed as a large compute node to

upstream Network

“System administrator friendly” System Interconnect Module

• Minimal system admin dependency on network admin for most compute node provisioning actions

• Can never default into a switch mode – protects against network misconfigurations

Simplify connectivity and interoperability challenges with a preconfigured interconnect module

IBM Flex System Fabric SI4093 System Interconnect Module Highlights

Base - 95Y3313, ESWAUpgrade 1 - 95Y3318, ESW8Upgrade 2 - 95Y3320, ESW9

Claim * Based on SI4093 base plus upgrade 1 and 2 list price x 2 = $53.194, HP Virtual Connect Flex-10/10D x 6 = $75,450 => 41.8% http://h30094.www3.hp.com/product.aspx?sku=10440121&pagemode=ca **SI4093 data sheet 64x10Gb ports (640Gb) vs HP VC 10 data sheet 26x10Gb ports (260Gb) => 2.1x - http://h30094.www3.hp.com/product/sku/10440121 SI4093 42 Downlinks + 22 Uplinks (64 10GbE) vs HP FEX 16 downlinks + 8 Uplinks (24 10GbE) => 2.6x - http://h18004.www1.hp.com/products/quickspecs/14138_div/14138_div.html



I/O In

fras

tru

ctu

re

Ethernet Connectivity

Simple Management◊

Optimized for performance ◊

“Pay as you grow” scalability ◊

Lowest TCO◊

Easy interoperability

IBM Flex System Fabric SI4093 System Interconnect Module

Overview•Easy to deploy simple connectivity– upgrade in order

Important Features:•Exceptional price/performance•FCoE convergence as a transit switch•Easy connectivity to reduce deployment time•Low Touch – default mode or VLAN mode•Easy interoperability with other vendors switches•No need for network administrators to manage•Warranty & software upgrade licenses match the chassis•Alternative to: Pass-thru, HP Cisco FEX or HP VC Flex

Base 10 x 10G

bE

SF

P+

#2 = 4x10G

bE

#1 = 2x40G

bE

1Gb

E M

gm

t

Total Ports

10Gb

Downlinks

10Gb

Uplinks

40Gb

Uplinks

Base System (95Y3313, ESWA)

14 10 0

w/ Upgrade #1 (95Y3318, ESW8)

28 10 2

w/ Upgrade #2 (95Y3320, ESW9)

42 14 2

1. Default transparent end host mode2. VLAN aware mode for multi-tenancy environments or environments

requiring more control of VLAN Layer 2 forwarding. Also enables FCoE FIPS Snooping Bridge support enhancing the local FCoE operation in the chassis.



Connectivity options and distances for the SI4093Networking Cable and Transceiver Questions

Will your 1Gb connections exceed 100m (less than 550m)? = 1Gb Short Range Transceivers + Optical Cable

Will your 10Gb connections exceed 5m (less than 300m)? = 10 Gb Short Range Transceivers + Optical Cable

Will your 40Gb connections exceed 3m? = 40Gb Short Range QSFP+ Trans+ Optical Cable

Will your 1 or 10Gb connections exceed 300m (less than 10km)? = Long Range Transceiver + Optical Cable

• DAC cables are ideal within a rack or adjacent racks (lower cost & power)• Transceivers are best for longer distances

• DAC cables are ideal within a rack or adjacent racks (lower cost & power)• Transceivers are best for longer distances

1GbE

10GbE

5 Meters

DAC/TwinAxOptical Cable

300 Meters

10GBase-SRTransceiver

100 Meters

550 Meters

10KM

RJ-45

1000Base-SXTransceiver

1000Base-LX Transceiver

Optical Cable Optical Cable

40GbE

3 Meters

QSFP+

QSFP+ MTP Optical Cable

100Meters

40GBase-SR Transceiver

1000Base-TTransceiver

Optical Cable

10GBase-LRTransceiver



Portfolio Positioning



Low latency/ high bandwidth apps

Telecommunication

Enterprise applications

high PerformanceScalability

VirtualizationCloud computing

Simple management

Simple connectivity and interop with

upstream network

Infrastructure Applications

(file/ print/ collaboration)

Flex System Ethernet Module Positioning

EN2092

1Gb Scalable Switch

10Gb UplinksFor easy

Transition

Ethernet, iSCSIsupport

1Gb Ethernet

CN4093Convergence within the

chassisConnect to existing LAN and

SAN networkScalable fabric

Easy transition to 40GbEL2/3 function

Enhanced Virtual Fabric for reduce I/O cost and complexity

EN4093RHigh performance 10GbE

connectionL2/3 function

Scalable fabricEasy transition to 40Gb

Enhanced Virtual Fabric for reduce I/O cost and complexity

OpenFlow SDN Support

SI4093Simple setup and management

Scalable fabricEasy transition to 40Gb

Virtual Fabric for reduce I/O cost and complexity

EN6131High performance 40Gb

Low latency

EN20921Gb connectivity

Easy transition to 10Gb

EN409110Gb connectivityUnmanaged device

NEW Aug 6thNEW Aug 6th

SI4093

Simple Low touch

ConnectivityModule

10Gb performanceScalable

40Gb uplinksEthernet, iSCSIFCoE support

NEW Aug 6thNEW Aug 6thEN4091

10Gb Pass thru

Unmanaged Device

Non blockingConnectivity to

UpstreamNetwork

Ethernet, FCoE

EN6131

40Gb Ethernet

Eighteen uplinkHigh bandwidth

Low latency

Ethernet support

CN4093

10Gb Scalable Converged

Switch

40Gb uplinks

Native FC portsFCF support for

Storage node

Ethernet, FCoE

EN4093R

10Gb ScalableSwitch

40Gb uplinks

Ethernet, iSCSIFCoE support

OpenFlow Enabled

10Gb iSCSI, EthernetFCoE(transit)

10Gb ConvergedIn chassis

40Gb Ethernet

Updated Aug 6th

Updated Aug 6th



Features 10Gb Converged Switch CN4093

10Gb Ethernet Switch EN4093R

10Gb System Interconnect Module

SI4093

10Gb Pass ThruEN4091

Summary FCoE Convergence Multi mode flexibility Preconfigured end host mode 1:1 direct connect

Layer 2/3 Support Yes Yes (No L3 in Easy Connect) L2 aware Pass thru only

Multi Tenancy (SPAR)

Yes Yes Yes N/A

Transparent / End host mode

Yes – Easy connect Yes – Easy Connect Yes – default profile N/A

FCoE/CEE support Yes Yes Yes Yes

FC ports Yes – Omni ports No No No

Spanning Tree Yes (No, disabled in Easy Connect)

Yes (No, disabled in Easy Connect)

No N/A

Max through – put* 1280Gbps 1280Gbps 1280Gbps 280 Gbps

Stacking Yes2 Yes No N/A

Interoperability Open Open Open Open

Network Virtualization

VMready, Virtual Fabric (vNIC2 or Unified Fabric Port*)

VMready, Virtual Fabric (vNIC2 or Unified Fabric Port)

Virtual Fabric (vNIC2) N/A

Max Ports base + upgrades

42x downlinks10 GbE†2x uplinks 10 GbE SFP+12x uplinks SFP+ Omni 2x uplinks 40 GbE QSFP+

42x downlinks 10 GbE† 14x uplinks 10 GbE SFP+2x uplinks 40 GbE SFP+

42x downlinks 10 GbE† 14x uplinks 10 GbE SFP+2x uplinks 40 GbE SFP+

14 x downlinks 10GbE

14 x uplinks 10GbE SFP+

Open Flow Support No Yes (v7.7) No N/A

SDN VE Support Yes Yes Yes Yes† This configuration leverages six of the eight ports on the CN4058 adapter available for IBM Power Systems™ compute nodes 2 - Hybrid mode only 2 CN4093 + 2-6 EN4093R

Feature Compare of IBM System Networking offerings for IBM Flex System



Decision Tree: Storage Connectivity Requirements

Connect to

storage?

Compute Node: Ethernet: •LOM •CN4054 (with SW upgrade for iSCSI HW offload•EN4132/EN4054 for Power nodes

Chassis: Ethernet: EN4093R* or SI4093

Compute Node: Ethernet: •LOM or CN4054•EN4132/EN4054 for Power nodesFibre Channel: FC3172 or FC3052 (8Gb)FC5022 (16Gb)

Chassis: Ethernet: EN4093R*/SI4093 Fibre Channel: FCXXXX

Compute Node: Ethernet: •LOM or CN4054 (with SW upgrade for CNA function)•CN4058 (Power)**

Chassis: Converged Switch: CN4093*

Compute Node: Ethernet: •LOM or CN4054 (with SW upgrade for CNA function)•CN4058 (Power)**

Chassis: Ethernet Transit: EN4093R*/SI4093

TOR Converged Switch: G8264CS or Cisco/Brocade

NAS / iSCSI FCoE

Separate Fibre Channel

FCoE to TORBroken out to

IBM, Brocade or Cisco

FC directly out of chassis to FC SAN switch or FCoE to

storage device

Chassis Top-of-RackBreak out of

FC



Technical Overview and Use Cases



Simplify chassis connectivity using IBM System Interconnect Module

Overview•Default profile enables transparent integration

• No Spanning tree• Built in logic for loop prevention• Minimal setup or management required

•Key operating modes 1.Transparent mode:

• Presents an end host, (compute node) view to the upstream network• VLAN communication between compute node NIC and edge switch runs

transparently through the module (VLAN agnostic operation)• Enables server to server communication for optimum performance (i.e. vMotion)• Provides traffic consolidation in the chassis to minimize TOR port utilization• Supports FCoE pass-thru to a transit switch or FCoE gateway device

2.VLAN aware mode (Local Domain)• Provides greater security and separation for multi-tenant environments• Enables forwarding decisions on client VLANs (ACL support)• Can re-use IP addresses in multi-tenancy configurations• Supports FCoE transit switch operation, FSB operation



Hardware Description

RJ45 Mgmt Port

Serial PortSFP+ Connector (1Gb/10Gb) QSFP+ Connector (10Gb/40Gb)



Switch Partition (SPAR) – How it works:1) Creates separate virtual switching contexts by segmenting the data plane of the switch.

2) The management and control plane partition of a physical switch is NOT addressed within the SPAR context and are still common across the partitions.

3) Implemented as a dedicated VLAN which is generally defined by a set of internal server ports and a single external uplink port or LAG (static or LACP).

4) Multiple uplink ports or LAGs are not allowed within a SPAR, eliminating the possibility of misconfiguring the device and creating a loop with the upstream network.

5) Port memberships are mutually exclusive between SPARs.

6) By default, a SPAR is a private domain that operates as a Layer-2 broadcast network.

7) Hosts on the same VLAN attached to a SPAR are allowed to communicate with each other and with the upstream switch.

8) Hosts attached to the same VLAN but on different SPARs don’t communicate with each other within the embedded interconnect. Traffic needs to be bridged by the upstream switch in this situation since there is no communication between SPARs within each broadcast domain.

9) SPAR can be configured to operate in two basic domain processing modes: Transparent and VLAN aware (Local).



SPAR support for SI4093 is a superset containing select features from SPAR Phase 1 & 3 plus customizations specific only to SI4093

Key differences in SPAR on SI4093

•Addition of “Host Mode”• Enables multiple SPARs to

connect to the same upstream network

• Disables MAC learning on upstream ports and applies ACLs to avoid MAC Flapping

•FCoE support within a SPAR context• Transparent mode support –

pass-thru• Local mode – Transit / FSB

•Increase of local VLAN support to 256



EXT-1

SPAR-1 (Pass-Through)Default VLAN-4081

Port: {INTA1, INTA2, INTA3, EXT1, EXT2}

SPAR-2 (Pass-Through)Default VLAN-4082

Port: {INTB1, INTB2, INTB3, EXT3, EXT4}

EXT-2

PacketData

C-VLAN10

S-VLAN4081

PacketData

C-VLAN10

INTA1 INTA2 INTA3

S-VLAN4081

PacketData

C-VLAN10

S-VLAN4081

PacketData

C-VLAN10

S-VLAN4081

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

EXT-3 EXT-4

PacketData

C-VLAN10

S-VLAN4082

PacketData

C-VLAN10

INTB1 INTB2 INTB3

S-VLAN4082

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

S-VLAN4082

PacketData

C-VLAN10

S-VLAN4082

PacketData

C-VLAN10

SI4093 Packet forwarding in Transparent Mode



EXT-1

SPAR-1 (Local-Domain)Default VLAN-4081

Port: {INTA1, INTA2, INTA3, EXT1, EXT2}

SPAR-2 (Local-Domain)Default VLAN-4082

Port: {INTB1, INTB2, INTB3, EXT3, EXT4}

EXT-2

PacketData

C-VLAN10

PacketData

C-VLAN10

INTA1 INTA2 INTA3

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

PacketData

C-VLAN10

EXT-3 EXT-4

INTB1 INTB2 INTB3

PacketData

C-VLAN10

PacketData

C-VLAN10

VLAN-10{EXT1, EXT2, INTA1, INTA2}

VLAN-20{EXT1, EXT2, INTA2, INTA3} VLAN-10

{EXT3, EXT4, INTB1, INTB2}

VLAN-20{EXT3, EXT4, INTB2, INTB3}

PacketData

C-VLAN10

SI4093Packet forwarding in VLAN aware Mode (Local Domain)



SPAR-1 SPAR-2

SI4093

vSwitch-1 vSwitch-2

ITE-1

vSwitch-1 vSwitch-2

ITE-2

vSwitch-1 vSwitch-2

ITE-3

DC Network

SI4093 Multi-Tenancy Host Mode



SPAR-1 SPAR-2

SI4093

vSwitch-1

ITE-1 ITE-2 ITE-3

DC Network

PG-1 PG-2

vSwitch-1

PG-1 PG-2

vSwitch-1

PG-1 PG-2

V10, V11, V12 V20, V21, V22

SI4093 Multi-Tenancy with VLAN aware

Each port group within vSwitch is using a specific uplink with VLAN separation among them. Each port group mapped to a SPAR so they can use separate uplink port set from the SI4093. In this kind of topology the SI4093 can run in VLAN aware mode and learn MAC address on uplink port. No dependency on centralize controller for server/VM MAC learning



Hypervisor

MGMT

Heart Beat

LiveMigration

StorageiSCSINFS

FCoE

Supervisor-SwitchHypervisor-Firewall

VM VM

P0 P1 P3P2

Hypervisor

Hypervisor

MGMT

Heart Beat

LiveMigration

StorageiSCSINFS

FCoE

Supervisor-SwitchHypervisor-Firewall

VM VM

P0 P1 P3P2

Hypervisor

Partition 1MGMT, Migration

Storage, Heart Beat

Partition 2VM Data

SI4093 1

INT1A

INT2A

INT14A

INT1B

INT2B

EXT1

INT14B

EXT2

EXT3

EXT4

EXT6

EXT8

EXT5

EXT7

Partition 1MGMT, Migration

Storage, Heart Beat

Partition 2VM Data

SI4093 2

INT1A

INT2A

INT14A

INT1B

INT2B

EXT1

INT14B

EXT2

EXT3

EXT4

EXT6

EXT8

EXT5

EXT7

Nexus 5K Nexus 5K

Flex Chassis

14 X ½ wide ITE

40 Gb 40 Gb40 Gb 40 Gb

14 X 10Gb

14 X 10Gb

14 X 10Gb

14 X 10Gb



Recommended configuration using end to end IBM offering

Adapter Integrated Device FCoE Top Of Rack Switch

SAN Switch Storage Target OS levels

LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes

SI4093 10Gb Module G8264CS (IBM Converged ToR) in NPIV mode

Cisco & Brocade SAN FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV

Win2008, ESX 4/5, RHEL 5/6, SLES 10/11

LOM or CN4054 (pNIC, vNIC

mode)

G8264CS

G8264CSV7000

SI4093 Flex System Fabric Interconnect

Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•IBM 10Gb System Interconnect Module aggregates all FCoE traffic and passes to LAN and SAN.

Black line is FCOERed line is Fibre Channel

Cisco or Brocade

SAN

IBM Converged ToR



Connecting to Cisco network: Customer already owns Nexus ToR

LOM or CN4054 (pNIC, vNIC mode)

V7000


Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•Connects to existing or new Cisco network.


Cisco SAN

Nexus 55XX

Nexus 55XX




SI4093 10Gb Module Nexus 5548 / 5596 Cisco SAN FC: V7000, SVC Win2008 R2SP1, ESX 5.0 U1, RHEL 6.3



Connecting to Brocade network: Customer already owns Brocade VDX or SAN

LOM or CN4054 (pNIC, vNIC

mode)

V7000


Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•Connects to existing or new Brocade network


BrocadeSAN

Brocade VDX

Brocade VDX




SI4093 10Gb Module Brocade VDX Brocade SAN FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV

Win2008, ESX 4/5, RHEL 5/6, SLES 10/11

© 2012 IBM Corporation

IBM Flex System EN6131 40Gb SwitchIBM Flex System EN6132 40Gb Adapter

Matthew Sheard - Mellanox

Aug 2013



Networking

IBM Flex System 40Gb Ethernet Switch (EN61310) and adapter (EN6132)N

etw

ork

ing

In

fras

tru

ctu

re

End to end 40Gb enables highest performance

End to end 40Gb enables highest performance

Ultra low latency for HPC, High Frequency Trading and

Enterprise Clients◊

Up to four 40Gb ports per single wide node

◊Highest performance non

blocking architecture drives up to 2.8TB bidirectional

throughput

• High performance end to end 40Gb connectivity with ultra low latency

• Non blocking throughput that is highly suited for High-Performance Computing, High Frequency Trading and Enterprise Data Centers.

• Enables clients to maximize compute utilization by removing IO bottleneck

• Reduces cabling by 4 is to 1 for applications requiring 40Gb throughput



Mellanox Value Proposition

Mellanox heritage in technical computing means that Mellanox switches have some of the lowest port to port latency and jitter, providing a stable high speed interconnect for todays demanding business and virtualization environments.

Financial

Cloud & Web 2.0

Clustered Database Weather Digital MediaComputational Aided

Engineering BioscienceLabs, ResearchUniversity, Academic Oil and Gas

Web 2.0 DB/EnterpriseHPC Cloud Financial Services

Storage



Performance and “Green” Too !

Efficiency: Reduce components and maximize productivity a data center.

Energy: Reduce power and cooling expenses

Complete High-Performance Scalable Interconnect Solutions for Server and Storage

TCO

50+%Reduction

Energy Costs

65+%Reduction

Infrastructure

60+%Saving

Performance

100+%Increase

Opex: Reduce management overhead and costly real estate.



RDMA Technology Becoming Mainstream

Remote Direct Memory Access• Direct access of one computer’s memory by another computer without involving

either ones operating system

• Allows applications to transfer data directly to the buffer of a remote application

• Zero-copy, I/O offload, CPU bypass technology for data transfer

• Near wire speed data transfers with sub 1us latency

Standard RDMA Protocols• RDMA-over-Converged-Ethernet (RoCE)

•Up to 40Gb/s

• InfiniBand: •Up to 56Gb/s

IBM:Yuqung Gao, Internet Centric ComputingWXS RDMA ICPE 2013



I/O Offload Boosts Application and Business Performance

~88% CPU Available for Application

Use

r S

pa

ceS

yste

m S

pa

ce

~53% CPU Available forApplication

~47% CPU Overhead/Wait

~12% CPU Overhead/Wait

Without RDMA With RDMA and Offload

Use

r S

pa

ceS

yste

m S

pa

ce

I/O Offload:More

Efficient Application Processing



Underlying Technologies Enabling Cloud 2.0

Cloud 2.0

Virtualization

Private Clouds

Cloud 1.0

Private Clouds

Overlay Networks

VXLAN, NVGRE

Interconnect Offload Engines

Reduce Application Cost

Open Platform

Open Ethernet, SDN

Scalability

Virtually Unlimited Tenants

Mobility

Resources Everywhere

Serviceability

Any Service, Any Time

Small Scale Clouds

Fixed Resource Allocation



Triple the number of Virtual Machines per server: from 20 to 60 VMs Higher storage density: 2X throughput and I/O Operations per Second (IOPS)

• Atlantic.net achieves 2X cloud I/O performance vs. Amazon with less cost using InfiniBand

Drive Higher Dollars Per Square Foot for Cloud

Without Mellanox With Mellanox

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM

VMVM



RoCE delivers lower latency, lower CPU load and higher bandwidth• Latency as low as 1us compared to 2.1us on iWARP• Up to 6X in Throughput (64B) compared to iWARP

iWARP requires TCP, using packet drops to handle heavy traffic• TCP is handled by the CPU or a special and complicated TCP Offload (TOE) NIC

RoCE requires NO CPU involvement

RoCE vs. iWARP



EN6131 Switch Features

Management plane Data plane• 10/100/1000 ETH RJ45 MNG ports• USB Console port for Management• Dual SW image • Static IP • DHCP • SSH • Telnet • RADIUS \ TACACS+ \ LDAP• NTP• FTP\TFTP\SCP• SYSLOG • Auto Temperature control • Events history • Port counters• SNMP v1,2,3• Web UI • In-band Management

• Port mirroring• sFlow• Rapid Spanning Tree (802.1w)• Flow control (802.3x)• DCB

• PFC (802.1Qbb)• ETS (802.1Qaz)

• LLDP• LAG/LACP (802.3ad)• 48K Unicast MAC addresses• Cables/transceivers identification• IGMP v1,2• VLAN (802.1Q) – 4K• ACL – 12K rules• Jumbo frames 9K



Leading throughput: 2.5X better

• 2.88Tb/s throughput on a single chip, running Full Wire Speed at any packet size

Leading L2 unicast/multicast latency for L2/L3 switches: 2X better

• 198-223ns for any packet size

Industry record power efficiency: 6X better than competition

• Sub 0.6Watt per 10GbE throughput with 100% load at Full Wire Speed

Shattering Ethernet Switching Performance Records

Packet Size (Bytes)

La

ten

cy

(n

s)

‘Zero’ Jitter!



Performance Driven Applications From Server to Fabric

Storage Connectivity Big data applications Distributed DB VM Environment (fast VM Migration) HPC Disaster Recovery Add more capacity to strained network

It is not about the number of servers you run

It is about how you run them

The Price of Building a Machine to Process 1TB of Data in 1 Minute

Migration Time Over the Network (sec)



E2E 40GbE Lowest Latency Cut through Solution

Superior message rate

Lowest Latency 40GbE in Trading

40GbE NASDAQ connectivity at the lowest latency

Exchange

MC streams

UC AckTCP/UDP(UC)

10.10.0.0/30Direct interface (pseudo vlan 2) 10.10.0.2

10.10.0.1

10.10.10/2Single VLAN

SX1036 TOR

Exchange Entry



End-to-End 40GbE Live in US-based Exchange Co-Location

“Mellanox provided a 40 Gigabit Ethernet solution that delivered high density, sustainable and high throughput, low latency and low power consumption that was beyond our expectations and in ways no other vendor could. We applaud the Mellanox team for their great support and helping us break new latency performance records.”Ido Rosen, quantitative researcher, Strike Technologies

Breaking New Latency Records



Mellanox ConnectX-3 Dual-Port 40GE NIC and Switch release in Q3/2013 Single wide chassis: 14x ITEs / Blade Servers support 2 adapters per server Double-wide chassis: 7x ITEs / Blade Servers support 4 adapters per server 14 ports Internal I/O; 18 ports External I/O Total System Compute up to 2.24Tbps System Front I/O up to 2.88Tbps

IBM Flex System Chassis with Mellanox 40Gb Interconnect

IBM PureFlex System



14 Compute Blades each using Single EN6132 Dual Port 40GbE NIC

2 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE

Compute I/O 1.12 Tbps @40 Gbps Uplink I/O up to 1.44Tbps @40 Gbps

Dual Star Architecture Dual-Dual Star ArchitectureMellanox Interconnect in IBM PureFlex

10GbE

10GbE

40GbE

40GbE

22 x 10GbE

18 x 40GbE

40GbE

40GbE

40GbE

40GbE

18 x 40GbE

18 x 40GbE

22 x 10GbE

18 x 40GbE

18 x 40GbE

18 x 40GbE

EN4093R

EN4093R

EN6131

EN6131

EN6131

EN6131

EN6131

EN6131

ITESWITCH

ITESWITCH

• 14 Compute Blades each using Dual EN6132 Dual Port 40GbE NIC

• 4 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE

• Compute I/O 2.24 Tbps @40 Gbps• Uplink I/O up to 2.88 Tbps @40 Gbps



Mellanox based technology is available for IBM Blade, Rack, & Flex Communities

Heritage of technical computing being broadly adopted:• Cloud, • Database & Analytics• Big Data• Storage

Core RDMA Technology Becoming Mainstream• Web 2.0, Cloud, Analytics, and Storage• Out of Box support for Linux, Windows, Cloud, & Hadoop Distros• Storage: Windows Server 2012 with SMB 3.0

IBM has Broad Mellanox based Product Offering Today• How can we help to make your business grow?

Summary … and Next Steps



Trademarks

Trademarks

The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: AS/400, DBE, e-business logo, ESCO, eServer, FICON, IBM, IBM Logo, iSeries, MVS, OS/390, pSeries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM

The following are trademarks or registered trademarks of other companies

Lotus, Notes, and Domino are trademarks or registered trademarks of Lotus Development CorporationJava and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countriesLINUX is a registered trademark of Linux TorvaldsUNIX is a registered trademark of The Open Group in the United States and other countries.Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.Intel is a registered trademark of Intel Corporation* All other products may be trademarks or registered trademarks of their respective companies.

NOTES:

Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.

All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

References in this document to IBM products or services do not imply that IBM intends to make them available in every country.

Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.

The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

august network announcement v3

Documents