PowerPoint Presentation

Energy-Efficient Solutions for 10Gbps EthernetYury Audzevich, Alan Mujumdar,Philip Watts, Andrew W. Moore

MSN 2012 workshopFriday, July 13th, 2012

1Introduction

Energy-efficiency of transmission systems is one of the key priorities with respect to the next generation of networking equipment.

Open questions:What is the power contribution of the lower layer transmission protocols?

What is the power impact of the encoding blocks?

Servers vs Network for a Google cluster (fat trees topology)

(Energy proportional datacenter networks, Abts et al. ISCA 2010)2The focus of the research

Which effect DC-balanced codes do have on the optical transmission system? the effect on optical power requirement? front-end power contribution, like PMA and PMD?the power consumption of line coding block itself? and in particular: what is

??????310Gb/s optical link simulations

Optical link transmission system:219 bits PRBS is used as an input, the baud rate is adjusted after encoding

Optical link parameters: 100m Single Mode Fibre with parameters satisfying requirements for 10Gbps Ethernet over SMF

Optical link receiving system:Optical receiver with direct detector and AC coupling achieved using High Pass FilterBER is calculated using the complementary error function410Gb/s optical link simulations (cont.)

The transmission system is relatively insensitive to the DC-balanced codec choice Taking 100MHz HPF cut-off frequency and assuming 20dB link budget, the laser power requirement is lower for encoded sequences (0.3mW of savings) in comparison to PRBS

betterbetterbetterbetter5Physical Coding Sublayer 8B10B (and 64B66B)

8B10B line code:Encoder/Decoder implemented 3B4B and 5B6B codes plus disparity control check for DC-balance

64B66B line code:Encoder/Decoder decoding from XGMII into 10GBASE-R formatScrambler/Descrambler mixing of data to avoid long sequences of 0s/1s

Codecs were implemented in Verilog HDL and Synthesized using 90nm and 45nm technical process librariesIndustry standard estimation tools were used for power measurements

6

Early-days results 8B10B PCS power

10Gb/s link results: obtained for 30 microseconds simulation periods, with a symbol clock frequency of 625MHz for both 45nm and 90nm tech. processIDLE sequences costs MORE to encode

Low leakage 45nm library provides decrease in power by a factor of 2Inverse ofenergy-proportionality7

Power estimates 64B66B PCS

10Gb/s link results: Identical pattern sent for 30 microseconds of simulation periods, with a symbol clock frequency of 156.25 MHzPower consumption of 64B66B codec is actually more data proportional

Scrambling & gearbox modules have a fixed power cost (~1.5-2.5 times larger power dissipation than the combination of encoder & decoder power)64B66B10GBASE-SR / 10GBASE-LR(commonly used)

8B10B10GBASE-LX4 (less common)8Physical Medium Attachment 8B10B and 64B66B

PMA components are built using both CMOS and MCML logic families

CMOS designs were synthesized using standard cell librariesMCML designs were built, optimized and analysed using HSPICE tools

9PMA power 8B10B and 64B66B

MCML power is independent of the operating frequency but is strongly related to the optimization criteria At high clock frequencies MCML designs become more power efficient than their CMOS counterpartsEven well power-optimized PMA designs may require 5x-10x times higher power than the corresponding PCS blocks!

64B66B

10Implications

Our recent analysis of realistic trace data(10Gbps) showed average link utilization of only 8.79% - in concordance with [1].The majority of the networks are overprovisioned to sustain peak loads and underutilized most of the timeCurrent implementations of Ethernet standards require continuous transmission of IDLE code words (even in the absence of MAC traffic)SoMay be, we need a system that has good energy-proportionality and can quickly restartSounds like we need a new MAC

[1] T. Benson, A. Akella, and D. A. Maltz. Network traffic characteristics of data centers in the wild. In Proceedings of ACM IMC '10, pp. 267-280, New York, USA, 2010.11

Remember this one? Ethernet CSMA/CD

Many features/ideas we dont want, but one we do:Or an old energy-efficient MAC

Preambles give clocks valuable re-sync. time and allow photonic systems to turn back on

12Energy-efficient MAC

Where do we get energy-savings from: Powering down the codecs when no data is present Using a synchronization preamble prior to data transmission for fast CDR

With avg. Ethernet frame size of 1150bytes and 64bits of preamble, the effective energy-saving is ~93%Is the protocol going to make a difference? YES, It makes the difference93%93%89%87%13Key take-aways

Optimal laser power is independent of the DC-balanced codec chosenCodec power consumption is not always data-proportionalSerialization/deserialization power dominates over all the other power groups New MACs (off when idle) do save significant powerHow do we test, build, trial?regular NICs dont helpNeed something programmable but FAST

Thank you!QUESTIONS?14