1

INFINIBAND

PREPARED BY:- SHAH

PULKIT .N.

2

WHAT IS INFINIBAND..?• Just as we all are aware of the Ethernet cable popularly know to us as a

Lan cable , similarly there is a fiber optical cable developed for the use in high-performance computing at places where it is to be dealt with large amount of data. Such kind of cables are know as InfiniBand.

• This is how a InfiniBand looks like. It looks same l like a normal Lan cable. But it’s advance feature f features makes it different from a Ethernet cable. c Its features include high throughput ,low latency l o , quality of service and scalable . failover. It is designed to b

3

• InfiniBand specification defines a connection between processor nodes and high performance I/O nodes such as storage devices.

• InfiniBand host bus adapters and network switches are manufactured by Mellanox and Intel .

• InfiniBand offers point-to-point bidirectional serial links intended for the connection of processors with high-speed peripherals such as disks. 

• The actual data transfer rate in InfiniBand varies from 2 Gbit/s for a normal 1x adapter to 120 Gbit/s for a 12x adapter.

4

HISTORY OF INFINIBAND• To overcome the slow data handling process InfiniBand (System I/O) were

developed.

• In 1999 merging up of two competing designs originated the InfiniBand.

Future I/O, developed by Compaq, IBM, and Hewlett-Packard.

Next Generation I/O developed by Intel, Microsoft, and Sun.

• As of 2009 InfiniBand has become a popular interconnect enterprise datacenters.

• In 2008 Oracle’s  HP Oracle Database Machine which utilized InfiniBand for all IO operations

• In 2009, IBM’s DB2 servers communicated with each other over an InfiniBand interconnect.

• In 2010, network storage manufacturers increasingly adopt InfiniBand for modern network storage

• In 2010, Oracle’s Exadata, used InfiniBand QDR with 40 Gbit/s (32 Gbit/s effective).

• In June 2011,infiniband switches and adapters were announced at the International Supercomputing Conference.

5

ARCHITECTURE OF INFINIBAND• Defines multiple devices for

system communication: channel adapter, switch, router, and a subnet manager

• At least one channel adapter is required for each end node

• Subnet Manager sets up and maintains link

• Channel adapters and switches contains Subnet Management Agent(SMA) for handling communication with the subnet manager

6

I/O ARCHITECTURE OF INFINIBAND• A switch-based serial I/O interconnect architecture.

• InfiniBand uses a switched fabric topology, as opposed to a hierarchical switched network like traditional Ethernet architectures. All transmissions begin or end at a "channel adapter." Each processor contains a host channel adapter (HCA) and each peripheral has a target channel adapter (TCA). These adapters can also exchange information for security or quality of service (QoS).

• It can be said that the switched fabric architecture used by InfiniBand is far better then shared bus architecture used by other data transmission devices.

7

InfiniBand uses switched fabric architecture and thus has the features shown above.it can handle large amount of data thus it is scalable, it switches to standby computer during fault, and is totally reliable.

Features Fabric Bus

Topology Switched Shared

Pin count Low High

No of end points Many Few

Reliability Yes No

Scalable Yes No

Fault tolerant Yes No

8

SWITCHED FABRIC I/O ARCHITECTURE

Switched fabric is a network topology where network nodes connect with each other via one or more network switches. Switched fabrics can offer better total throughput than broadcast networks because traffic is spread across multiple physical links. Fiber Channel, Is a high-speed network technology (commonly running at 2-, 4-, 8- and 16-gigabit per second rates) primarily used to connect computer data storage.

9

INFINIBAND CABLE AND COMPONENTS

1X

2.5 (5/10)

Gbs

4X

10 (20/4

0) Gbs

12X

30 (60/120)

Gbs

Parallel Optics IB 4X Connector

Optical to Copper Smart Module

4X Copper connector

10

LAYERS OF INFINIBAND

11

PHYSICAL LAYER LINK LAYER

• Defines both electrical and mechanical characteristics for the system

• Includes cables and switches for fiber and copper media, backplane connectors.

• Defines three link speeds, 1X,4X,12X

• Each individual link is a 4-wire differential bi-directional connection that provides a full duplex connection at 2.5 Gb/s

• Encompasses packet layout, point-to-point operations, and switching within a local subnet

• Packets- Two types: Management and Data packets

1. Management Packets- Used for link configuration and maintenance

2. Data Packets- Carry up to 4k bytes of a transaction payload

• Switching- Devices within a subnet have a 16-bit Local ID assigned by the subnet manager. This LID is used for addressing.

12

NETWORK LAYERTRANSPORT LAYER

• Handles routing of packets from one subnet to another(within a subnet a network layer is not required)

• Packets contain a global route header(GRH)

• GRH contains the 128-bit IPv6 address for the source and destination of the packets.

• Responsible for in-order packet delivery, channel multiplexing and transport services

• Also handles transaction data segmentation when sending, and reassembly when receiving

13

PROS & CONS OF INFINIBAND

• Superior performance

• Low-latency

• High-efficiency

• Fabric consolidation and low energy usage

• Reliable, stable connections

• Data Integrity

• Highly interoperable environment

• Complex in design

• Few platforms supports it as yet

• Bleeding edge, for now, so users will need to perform extensive testing

14

CONCLUSION

• Greater processor performance

• Supports RAS (Reliability, availability, Serviceability)

• Works both in-the-box and enables Bandwidth Out of the Box

• Supports scalability

15

REFERENCES

1. InfiniBand Trade Association www.infinibandta.org

2. InfiniBand Network Architecture, MindShare, Inc. By Tom Shanley

3. InfiniBand Architecture, Development and Deployment: A strategic guide to server I/O

solutions By William T. Futral

4. QoS in InfiniBand Sub networks:

Alfaro,F.J.; Sanchez, J.L.; Duato, J.; Department de Informatica, Univ. de Castilla-La Mancha, Albacete, Spain

16

THANK YOU