SWIFT: A High-Capacity Wavelength-Striped Optically-Switched

Interconnect

Michael Dales, Madeleine Glick

Intel Research Cambridge

Project overview

The SWIFT work is part of an interdisciplinary DTI funded project. It’s a collaboration between:

• Intel Research

• University of Cambridge

• Essex University

• Intense Photonics

Overview

Motivation

SWIFT architecture overview

Media Access Control issues

SWIFT testbed description

Summary

Motivation

Motivation

Research over recent years has suggested applying optical networks to short range networks:

• Device interconnects, Cluster/supercomputer interconnects, Storage Area Networks, Application specific LANs

• c.f., Infiniband, Fibre-channel, PCI-Express

Aim: to provide high bandwidth (>= 100 Gbps) with low latency and low power

Motivation

There are a lot of exciting optical technologies in the long-haul domain we might like to apply, either existing or from research:

• Wavelength Division Multiplexing (WDM) to allow multiple channels over a single fibre

• Optical switching research to remove the expense/delay associated with OEO conversion

We would like to apply these kinds of technology to the short range interconnect domain

Challenges

Long haul equipment is not designed with ease of use or small configuration in mind

• Temperature controlled machine rooms

• Infrequent network configuration

• Manual set-up and tuning

No optical memory

• Fibre delay lines only offer a fixed delay

• Variable delay optical components (e.g., photonic crystals) still a way off

We want to replace this…

…with something like this

Architecture

Architecture overview

The SWIFT Architecture is our design for a packet switched optical interconnect based upon:

• Using WDM to increase bandwidth per link

• Having an all optical data path

• A single switch to simplify the design (currently)

• An electronic control plane to manage the optical domain

Using WDM

Traditionally WDM is used to allow a single fibre to carry multiple independent channels

In SWIFT we use a technique called Wavelength Striping to split a single data channel over many wavelengths

This increases bandwidth/reduces latency for packets

In SWIFT n-1 wavelengths are used in this way, with one reserved for control signalling

Overview

Switch design

The switch is split into two parts: optical data plane and electronic control plane.

Data will travel edge-to-edge optically, with no electronic processing.

Lightpaths need to be constructed before data sent into the network

• Asynchronous control channel used to communicate between nodes and switch over reserved wavelength

• No header processing on data packets

Switch fabric

Many possible technologies for building optical switching fabrics

• Mechanically moved mirrors

• Thermally switched device

• Semiconductor based devices

Which is right depends on what you want to do…

Switch fabric built with opto-electrical devices to provide suitably fast response to switching

• We currently use SOAs, which respond in a few nanoseconds

Switch fabric

Demonstrated switching 10 * 10 Gbps channels through an SOA (ECOC 2005)

55us/div – Packet is 94.72us data, 1.28 us guard band

Host interface

The host interface on the network has two main tasks:

• Manage the splitting of packets into wavelength striped form and back again

• Communicating with the switch over the control plane to request lightpaths

Media access issues

Lack of Optical RAM means conventional packet contention resolution cannot be done

• Can’t just “fire and forget” packets

Contention needs to be resolved *before* packets are injected into the network

Want to achieve packet switching

Obvious parallels here to circuit switching/OBS

Limited domain means our choices different

Media access issues

Basic protocol is just a request/grant protocol:

• Hosts request a lightpath to send a packet to a host

• Switch allocates a time for this to occur

• Notifies host when it can transmit

This, whilst sufficient has a high latency cost

Will be investigating techniques to improve on this:

• Reservation for predictable traffic by application?

• Random/weighted grants during idle time?

• Statistical pre-allocation based on recent history?

Testbed

Demonstrator

In addition to fabric experiments, we have been building a full testbed with real hosts.

Aim is to allow us:

• to examine the issues of building a full implementation

• run real applications and real traffic over the network

• provide based data for simulation models

• feel a lot of pain… ;)

Does not run as fast as the aim – making a slower version is hard enough.

Testbed overview

Built a 3 node test-bed

Two main components: host interfaces and switch

Control electronics on FPGAs

2 data in 1500nm range

1 control in 1300 nm range

Couplers/AWGs used to combine/split s

Current demonstrator

Current setup seen here

Three racks:

• 1: switch

• 2: host interface board

• 3: host interface transceivers

PCs off shot

Large due to using off the shelf components!

Demonstrator status

Demonstrator up and running

• SOA switch fabric working between nodes

• Data striped over multiple wavelengths

• Linux workstations plugged in talking over whatever (TCP, UDP, …)

Undergone a bit of a redesign due to some hardware issues

• Hard to benchmark TCP when you lose too many packets…

• Clearly hardware vendors hate documentation as much as softies

• Beware what you’re getting yourself into when you build a network :)

New version (as of Monday) much more reliable, so hopefully results soon…

Related and future work

Current/future work

With SWIFT we have some ongoing and future avenues of research:

• Photonic device control – plug & play photonics

• Switch fabric design – how to best make the fabric

• Media access control – how to make it all work efficiently

Meta-research issues:

• How does one test a new network in multiple domains?

• Lots of ad-hoc solutions, mostly aimed at Internet traffic

• Finding traces to work from (as ever) a pain

• Unsexy science

Line encoding for wavelength striping

Naïve approach to multi-wavlength coding: n * 8b10b

However, we are worried about several issues in the network that better line coding could help with:

• Per-packet clock recovery

• Power balancing through the SOAs

Working with Computer Lab to apply a mixture of techniques to produce better line encoding:

• Clock recovery over multiple wavelengths

• Asynchronous coding to allow for early data recovery

• Better codes across wavelengths to balance power

Chip being built to allow high speed experimentation for these

Protocol specificatoin

Working with Peter Sewell et al in Computer lab with their protocol formalisation work

• Provide formal HOL model of protocol rather than woolly RFC

• Can verify traces from simulation/implementation against spec

• Can prove properties about network spec (e.g., approximates FIFO)

Previously they looked at sockets/TCP in an after fact fashion, now we’re providing a test case for doing it at design time

• Forces designer to be specific early on

• Process of writing formal spec itself highlights gotchas

Now have 43 pages of HOL to describe basic SWIFT MAC

Summary

Summary

We’re interested in applying optically-switched networking to the problem of next generation interconnects

Proposed the SWIFT architecture, which uses a mixture of optics and electronics to provide a high-capacity interconnect

Identified some interesting areas for further research

Questions?

JB?