deliverable d2.2 investigation and assessment of the ... d2.2.final.pdf · for the flexi-grid...

STRAUSSScalable and efficient orchestration of Ethernet services

using software-defined and flexible optical networksDeliverable D2.2

of 78

Deliverable D2.2

Investigation and assessment of the potential technologies

for the flexi-grid optical path-packet Infrastructure for

Ethernet Transport

Status and Version: Version 3.0

Date of issue: 22/07/2015

Distribution: Public

Author(s): Name Partner

Shuangyi Yan

Yan Yan

Georgios Zervas

Dimitra Simeonidou

UNIVBRIS

Michela Svaluto Moreolo

Laia Nadal

Josep M. Fabrega

CTTC

Michael Schlosser

Luz Fernandez del Rosal

Kai Habel

HHI

Yuki Yoshida

Ken-ichi Kitayama

OsakaU

Masato Nishihara

Toshiki Tanaka

Tomoo Takahara

Jens C. Rasmussen

Fujistu



of 78

Victor Lopez

Felipe Jimenez

TID



of 78

Executive Summary

This deliverable presents the work performed by STRAUSS project during its second year (Y2) in

terms of investigation and assessment of the potential technologies for the flexi-grid optical path-

packet infrastructure for Ethernet transport. Based on the previous requirement analysis in D2.1, the

key technologies, including software-defined sliceable transceiver, flexible/adaptable optical nodes,

variable capacity OPS and the integrated OPS/OCS interface, are investigated and assessed by

simulation and experimental verification. The most potential solution or technologies are selected to

enable the flexi-grid optical path-packet infrastructure. This document provides a description of the

potential technologies and the related test results.

In OPS domain, L-VC OPS network based on the OFDM technologies have been investigated and

some practical adaptive-modulation technique, such as for designing the cross-talk tolerant packets,

have been proposed. Furthermore, application throughput and latency of the distance-adaptive

DMT-based OPS network have been analyzed.

In regard to the Software-defined sliceable transceiver, OFDM technologies have been investigated

to enable 400Gb/s optical Ethernet scenario. A 400Gb/s DMT transponder prototype have been

implemented and demonstrated successfully. For the implementation of the sliceable OFDM

transceiver, an OFDM-based BVT, able to generate a single multi-format rate/distance adaptive

flow, has been studied and optimized. It can be used as S-BVT building block for cost-sensitive

applications, targeting the metro/regional network segment and inter-DC communication.

Rate/margin adaptive algorithms have been developed and integrated within the DSP modules of the

BVT to enable bit and power loading of the individual subcarriers for rate/distance adaptive

transmission according to the capacity request and channel estimation. The real-time implementation

of the OFDM-based transmitter have been developed and tested. Detailed research have been carried

on to investigate the system performance by means of numerical simulation and experimental

demonstration. In addition, a 40Gbaud QPSK/16QAM multi-format transmitter is developed to

enable spectral-efficient transmission in OCS-based Core optical networks.

With regards to optical node design, flexible/adaptable optical nodes is developed based on AoD

concept for flexi-grid DWDM networks. The node composing algorithms are used to synthesis and

reconfigure the optical nodes, while the optical monitoring modules monitor the synthesized optical

node and guarantee it work correctly. The proposed AoD-based optical nodes deploy network

functions according to the traffic requests, which will improve hardware utilization efficiency. For

the prototype implementation, we have achieve several optical node functions, such as architecture

programmable ROADM, software-defined programmable transmitter. The software-defined

programmable transmitters provide interfacing between OPS and flexi-grid OCS domain for

extremely large optical capacity.

The integrated OPS/OCS interface, which is developed with FPGA develop board (HTG-V6HXT-

X16PCIE board), provides two hybrid ports (OCS/OPS traffic) and two OCS-only ports. The hybrid

ports are used to connect to the OPS domain, while the OCS-only ports are to the OCS domain. The

OPS-OCS integrated interface convert OPS traffic to OCS domain and forward traffic to high baud

rate transmitters in OCS domain for spectral efficiency transmission.



of 78

Table of Contents

Executive Summary 3

1 Introduction 6

1.1 End-to-end Ethernet Networking 6

1.2 Ethernet transmission in OPS domain 8

1.3 Ethernet transmission in OCS domain 10

2 Software-defined Sliceable transceiver 10

2.1 Sliceable OFDM Transceiver 10

2.2 DMT transceiver 17

2.2.1 Comparison of modulation techniques for 400 Gb/s Ethernet transceiver 17

2.2.2 Development of 400G(4x100G) DMT transceiver demonstrator 19

2.3 Spectral-efficiency transmission in EON with high-baud rate multi-format transceiver 20

2.3.1 High-baud rate multi-format transceiver 20

2.3.2 Impairment mitigation with receiver-side digital signal processing 22

2.4 Real-time OFDM Transmitter with Ethernet MAC 24

2.4.1 Real-time implementation 24

2.4.2 Real-time MAC Layer design 25

3 Flexible/adaptable optical nodes for flexi-grid DWDM networks 33

3.1 Development of key subsystem 34

3.1.1 Software-defined programmable superchannel transceivers 34

3.1.2 Programmable ROADM 37

3.2 Cost and energy consumption analysis 38

4 Variable capacity OPS based on sliceable transceivers 42

4.1 Single-carrier payload packet versus Multi-carrier payload packet 43

4.2 Cross-talk tolerant multi-carrier payload packet 44

4.3 Distance-adaptive IM-DD FL-VC OPS based on DMT 47

4.4 Performance analysis of distance-adaptive FL-VC OPS network 49

4.4.1 On packet capacity distributions 49

4.4.2 Achievable throughput by fixed-length optical packets 50

4.4.3 Achievable throughput by the distance-adaptive FL-VC packets 53

4.4.4 Performance analysis in MAN 53

4.4.5 On the designing of distance-adaptive FL-VC OPS network 56

4.4.6 Latency considerations in FL-VC OPS network 60



of 78

4.4.7 Energy consumption of intra-datacenter OPS networks 63

5 FGPA-based real-time OPS/OCS integrated interface 67

6 Conclusions 71

7 List of acronyms 72

8 References 76



of 78

1 Introduction

STRAUSS project aims to control and manage a network scenario covering multiple technology

domains and provision end-to-end beyond 100Gbit/s Ethernet services. Figure 1 shows the four

layers in STRAUSS architecture. This architecture is proposed as a realistic solution for Ethernet

transport scenarios enabled by Software Defined Networking.

Figure 1: The four layers of STRAUSS architecture

The data plane solution for STRAUSS project is based on two transport network technologies flexi-

grid and optical-packet switched. Thanks to both technologies several use cases are supported in the

project. The developed infrastructure covers different/heterogeneous technologies based on: i)

optical packet switching technology to provide scalable and cost/energy-efficient traffic grooming at

sub-wavelength granularity, ii) optical spectrum switching technology to provide flexible spectrum

management capabilities, and iii) software-defined and sliceable (multi-flow) bandwidth-variable

transponders (BVT) supporting multiple data flows with different modulation formats and bit rates.

1.1 End-to-end Ethernet Networking

As the optical Ethernet technologies provide network versatility in a simple, speed, reliable and cost-

efficient manner, it has gained great utilization in different network segments. So far, the optical

Ethernet has been deployed over fiber for short-range communications, over resilient packet ring to

provide a flexible, multi-service implementations span over thousands of kilometers for metro

networks [Ramamurti01], and even over flex-grid-based elastic optical core networks.

Virtual Transport Infrastructure 1 Virtual Transport Infrastructure n

...

...

SDN-based Service and Network Orchestrator

Network Control & ManagementNetwork Control & Management

Flexi-grid OCS

Domain 2Flexi-grid OCS

Domain 1

OPSOPS OPS/OCS

(BVT)

OPS/OCS

(BVT)

Virtual Resources Pool

GMPLS OpenFlow

Tra

nsport

Infr

astr

uctu

reT

ransport

Virtu

aliz

ation

Virtu

al In

fr.

Ctr

l &

Mgt

End-t

o-e

nd

Orc

hestr

ation

Virtualization Visor (Abstraction, Partitioning, Composition)



of 78

The STRAUSS project aims to define a highly efficient and global multi-domain optical

infrastructure for Ethernet transport. The multi-domain optical infrastructure covers optical packet

switching network (OPS) and optical circuit switching (OCS) networks using flexi-grid

technologies. In such a multi-domain, multi-technology network, software defined networking

(SDN) principles could provide network orchestration and enable an end-to-end optical transport

service [Yoshida14]. In terms of the data plane, “Layer 2” Ethernet service is provided in all the

optical segments. The multi-domain optical infrastructure aims to enable end-to-end Ethernet service

delivery on a global scale, which offers some attractive advantages in terms of service provision and

network management.

End-to-end Ethernet communications are a critical feature for variety applications. Inter-DCN

communications, as an example, need a large scale Ethernet service to bridge two geographically

distributed DCNs. Recently data center networks (DCNs) have experienced a remarkable growth

both in scale and in numbers of new deployments. Ethernet services are widely used both for intra-

and inter-DCN communications. In future, the large-scale network services will require several data

centers work together to provide smooth user experiences. Thus the soaring inter-DCN traffic will

require, not only large-capacity optical links connecting remotely located DCNs for replication and

partitioning, but also possible server-to-server cross-DCN communications. The direct server-to-

server cross-DCN connections enable the geographically distributed DCNs to appear as one big data

center, which could enhance scalability and minimize latency and cost. Under such application, the

end-to-end Ethernet services should be provided in a large scale over multiple-domain and multi-

technology network.

The main framework of STRAUSS project provides an efficient network infrastructure for Ethernet-

based multi-domain communications with capability to deliver Ethernet service globally. Figure 2

shows the main infrastructure for multi-domain communications provided by STRAUSS project.

The OPS architecture proposed in STRAUSS aims to provide the granularity and dynamicity

required for aggregation. The aggregated traffic is sent through the flexi-grid DWDM based core

networks and OFDM-based regional metro networks for spectral efficiency transmission. With

developed OPS/OCS interface, OPS domain is connected to the OCS domain, to jointly provide

multi-domain end-to-end Ethernet transmission.

In D2.2, the selected key technologies of the developed multi-domain, multi-technology network

infrastructure in the ICT STRAUSS project is reported to deliver Ethernet services over several

testbeds for multi-domain communications.



of 78

Figure 2 Use case for end-to-end transparent optical networks: multi-domain communication in the ICT

STRAUSS project

1.2 Ethernet transmission in OPS domain

The analysis of Big Data, e.g. based upon real-time complex event processing (CEP), will be

performed by exchanging huge quantities of data on inter-DC and intra-DC networks. The size of

data sets that are feasible to be processed in a reasonable amount of time is often limited by not only

the processing capability but also the bandwidth and the latency of data transfer over the networks.

Therefore, a huge bandwidth as well as low latency will be required for the network.

As the total fibre transmission capacity increases, the technologies for dividing the capacity into

many flexible paths are also evolving. One such technology is packet optical transport system,

where electric packet switches are integrated with WDM optical cross-connects to create flexible

sub-wavelength paths with arbitrary bandwidths. Packets provide sub-wavelength “logical” paths,

while a wavelength serves as a physical path.

In order to save energy, CAPEX and OPEX, a novel network architecture is needed to overcome the

limitations of the current router-based networks. There has been a transition or departure from

legacy IP over DWDM to simplified layer-2 switching. Layer-2 switches such as Ethernet switches

are in widespread use in metro-access networks. Beyond 100 Gb/s, however, the electrical switch

cannot be as efficient as the optical counterpart due to large forwarding delay and poor power

efficiency. An innovative approach to layer-2 will be needed to improve the power consumption and

packet forwarding delay of electrical switches.

The enabler will be photonic layer-2 (P-L2) networking. The key concept of the P-L2 is to simplify

the Ethernet bridging or switching functions as much as possible, by exploiting the inherent nature

of photonics and optics such as its high speed, abundant bandwidth, and all-optical processing

capability. The intention is to create a new photonic-native data transport protocol, not necessarily



of 78

emulating the conventional Ethernet protocol, while retaining the interface with the existing

protocols at the edges.

In Figure 3 the architecture of P-L2 network is compared with that of an Ethernet. A typical Ethernet

is a multi-hop network of Ethernet bridges or switches, in which all data packets are processed by

electrical circuits for the physical (PHY) and media access control (Eth-MAC) layers (Figure 3(a)).

In the P-L2 architecture, all-optical signal processing is adopted and combined with the digital

electronics in such a way that the best of both can be fully exploited (Figure 3(b)). The PHY layer at

the P-L2 edge converts the data signals from the PL2-MAC layer into optical signals and transmits

them to a P-L2 bridge over transparent optical channels with a flexible bandwidth of up to one Tb/s.

P-L2 bridges transfer the optical data signals from P-L2 edges to their destinations through optical

switches without O-E-O. At the P-L2 edge, the PL2-MAC layer converts the data packets of the

existing protocols such as Ethernet into data signals for P-L2. By employing the SDN technology,

address resolutions are managed from the outside. For the P-L2 bridge, optical packet switching

using high-speed space switches or high-speed tunable lasers are used to fully utilize the optical

transmission capability and to realize data aggregation or distribution in the optical domain. P-L2

can divide the bandwidth of each wavelength into many logical paths with narrower bandwidths

without using external electrical packet switches. P-L2 enables network virtualization which differs

from the legacy virtualization, outnumbering slices from 1 Gb/s to 100 Gb/s.

One major obstruct to realize P-L2 based on the OPS technology is the lack of readily available

optical RAM buffer. At present, fiber delay line (FDL) buffer seems to be a practical solution

despite its inflexible buffering capacity. A fixed-length (fixed time slotted) optical packet would be

preferable in FDL because of simple scheduling algorithm. Meanwhile the incoming Ethernet

frames have variable-length, .e.g., 64 to 1518 bytes (or 9600 bytes). In STRAUSS project, by

exploiting the bandwidth flexibility of the optical multicarrier modulation technique, such as OFDM

and DMT, we investigate an efficient design of fixed-length variable-capacity (FL-VC) optical

payload packet towards real-life P-L2 based on the OPS technology.

Figure 3: (a) Legacy multi-hop Ethernet switch network, and (b) Photonic layer-2 network



of 78

1.3 Ethernet transmission in OCS domain

Current core networks are using the technique of Dense Wavelength Division Multiplexing

(DWDM). OCS provides an efficient way to configure signals at a granularity of wavelength.

wavelength. Optical Transport Network (OTN) technologies have been widely used for long haul

transmission with digitally client signal wrapping and extra connection monitoring [Walker].

However, the OTN technologies, are not always required in core networks. Deploying Ethernet

technology over DWDM saves both CAPEX (capital expenditures) and OPEX (operation

expenditures). In addition, “Layer 2” Ethernet offers some attractive advantages to both

Infrastructure Carriers and Telecoms Service Providers in terms of service provisioning, the inherent

setup complexities of Managed Router networks being largely eliminated.

In STRAUSS project, we are targeting for an End-to-End Ethernet connection, which would allow

seamless transport of network services. The potential application of inter-DCN communication also

simplifies the link configurations with limited quantity of inter-connection links. The “Layer 2”

Ethernet will be the promising solution for STRAUSS project in core networks.

On the other side, the Ethernet over OCS or Ethernet over DWDM faces challenges to handle the

ever-increasing Ethernet traffic. Firstly, the Ethernet traffic will be highly variable, complex and

unpredictable. The varied requirements of different applications, changes in customer behaviors,

uneven traffic growth, or network failures will lead to the uncertainty in traffic demands,

granularity, and geographic and temporal distributions. The dynamic nature of Ethernet traffic

requires flexible optical node design for variable bandwidth provision. Secondly, the flow size of

Ethernet traffic has a large range from several Mbits/s to over TBit/s [Web14]. Thus the optical node

should support a large range of switching granularity. In addition, multi-dimensional technologies

will be used in optical networks to provide adequate optical bandwidth for emerging Ethernet

applications. The multi-dimensional optical networks, e.g., SDM/WDM/TDM, will require new

methods to allocate network resources for better resource utilization.

To satisfy the aforementioned requirement, the OCS network requires evolutions of several data

plane technologies. The key enabling technologies include software-defined sliceable BVT

technology, and flexible/adaptable optical nodes for flexi-grid DWDM networks. The key

technologies for OCS in STRAUSS will be discussed and demonstrated in Section 3 and Section 5.

2 Software-defined Sliceable transceiver

2.1 Sliceable OFDM Transceiver

OFDM technology arises as a suitable solution to design adaptive/programmable, software-defined

sliceable bandwidth variable transceivers (S-BVT). In fact, thanks to its subwavelength-granularity,

electrical OFDM subcarriers can be individually mapped with different modulation formats and

different power values can be suitably assigned to each of them. Thus the system can be efficiently

adapted to the optical path to be supported and to the traffic demand. Figure 4 shows an example of

S-BVT comprising an array of N DSP blocks; each of them is able to generate a data flow of



of 78

variable capacity and bandwidth occupation. After digital to analogue conversion and optical

modulation, performed by the Digital-to-Analog Converter (DAC) and the tunable front-end array,

respectively, the N data flows are aggregated and filtered at the Spectrum Selective Switch (SSS).

Targeting the metro/regional network segment, the optoelectronic front-end at the receiver is based

on cost-effective Direct-Detection (DD).

Figure 4 Sliceable BVT schematic.

In this deliveralbe, we focus on the design of a single BVT, which can be used as S-BVT building

block, able to generate a single multi-format rate/distance adaptive flow. Here we provide the

description of the transmission system, including the characterization of the most limiting devices.

We analyze the performance of the proposed system by means of numerical simulations and provide

preliminary experimental results.

The transmission system is shown in Figure 5. There, a simplified optoelectronic front-end at the

transmitter is considered, using a single DAC, a Tuneable Laser Source (TLS), a linear driver and an

external Mach-Zehnder Modulator (MZM). At the receiver side, a simple PIN photodetector and an

Analog-to-Digital Converter (ADC) are required. We propose to use Single Side Band (SSB)

OFDM in order to enable a more robust transmission against chromatic dispersion, compared to

double sideband OFDM or DMT, and thus extend the achievable reach. SSB modulation is

implemented by using an optical filter at the MZM output. Since the proposed BVT is designed as

the building block of a multi-flow S-BVT, in the integrated design, the SSS can act as both,

aggregator and SSB filter, without requiring any additional optical filters at the transmitter side. In

typical SSB systems, a guard-interval, equal to the OFDM signal bandwidth, is created between the

optical carrier and the signal itself, by means of DSP or analogue radio frequency (RF) upconversion

[Schmidt08]. We adopt digital RF upconversion in order to create both the guard-interval and a real

OFDM signal to be converted to analogue by a single DAC. The MZM working point has been set

near the null in order to achieve full intensity modulation. An example of the transmitted and photo-

node-1

node-N

TLSn &

MZMn

DACnDSPn

(S)-BVRxTLSN &

MZMN

Optical

Network

. . .

TLS1 &

MZM1DAC1DSP1

DSP-enabled programmable S-BVTx

DACNDSPN

. . .

DACarray

Multicarrieradaptive Tx-DSP

TunableFront-end

Aggregator &(SSB) filtering

Preamp

Rxn

Preamp

RxN

. . .

Preamp

Rx1

DSP-enabled programmable S-BVRx

. . .

DD Front-end

Optical flowdistributor

DSPnADCn

DSP1ADC1

DSPNADCN

ADCarray

Multicarrieradaptive Rx-DSP

node-2

BVRx



of 78

detected SSB signal, using this scheme, is depicted in Figure 6 (a) and (b), respectively. As seen in

Figure 6 (b), all the intermodulation products fall inside the guard-interval, enabling correct photo-

detection. An alternative SSB scheme is investigated in order to efficiently allocate the OFDM

signal in the available spectrum, as shown in Figure 6 (c) and (d). For this purpose, no guard-interval

SSB modulation is proposed, trading off spectral efficiency and performance. In this case, the MZM

works at the quadrature point to limit the intermodulation products due to the photo-detection

process [Svaluto12].

Figure 5 Optical system model and experimental set-up.

The proposed BVT transceiver can be adaptively reconfigured to occupy variable bandwidths in

order to transmit/allocate distinct data flows according to the target data rate/performance.

Two different SSB signals of 10 GHz and 20 GHz electrical bandwidth occupancy have been

considered using the two aforementioned schemes: SSB and no-guard interval SSB. Depending on

the transmitted signal bandwidth, two different DACs are considered. We have analyzed the system

performance using a high-speed DAC working at 64 GSa/s with 13 GHz bandwidth and an arbitrary

waveform generator (AWG) working at 20 GSa/s with 10 GHz bandwidth.

MZM

AWG

IFFT

OFDM BVTx DSP

Input data

.

.

.

.

.

.

.

.

.Ad

d C

P

S/P

.

.

.Map

per

Ad

d T

S...

P/S

.

.

.

Up

con

vers

ion

DA

C

Equ

aliz

er &

TS

rem

ova

l

OFDM BVRx DSP

.

.

.

.

.

.

.

.

.

S/P P/S

.

.

.

FFT

Dem

app

er

AD

C

TLS

CP

rem

ova

l

.

.

.

.

.

.

Do

wn

con

vers

ion

SSB filter BPF

OSA

EDFA EDFADriver

PIN

Optical channel

VOA50km 50km 50km

AWG DPO



of 78

Figure 6 (a) Transmitted and (b) photodetected SSB spectra. (c) Transmitted and (d) photode-tected no

guard-interval SSB spectra.

The DACs have been first characterized at CTTC Laboratory. Then, their impulse responses have

been modelled with 2nd order digital Gaussian filters of 20 GHz and 11 GHz, respectively, as

depicted in Figure 7 (a) and (b). In particular, in Figure 7 (a), the DAC transfer function cannot be

clearly identified at high frequencies (greater than 14GHz), due to characterization measurement

effects. In fact, the oscilloscope has been used at the 20 GHz bandwidth, obtained enhancing the

16GHz hardware bandwidth, using the DSP integrated in the equipment, distorting the acquired

DAC response.

Figure 7 (a) High-speed DAC and (b) AWG impulse response modelling.

-40 -30 -20 -10 0 10 20-40

-30

-20

-10

0

10

20

30

40

50

60

70(a)

Frequency (GHz)

Po

we

r (d

B)

-40 -30 -20 -10 0 10 20 30-40

-20

0

20

40

60(b)

Frequency (GHz)

Pow

er

(dB

m)

-40 -30 -20 -10 0 10 20 30-40

-20

0

20

40

60

Frequency (GHz)

Pow

er

(dB

m)

(c)

-40 -30 -20 -10 0 10 20 30-40

-20

0

20

40

60(d)

Frequency (GHz)

Pow

er

(dB

m)

0 5 10 15 20 25-100

-80

-60

-40

-20

0

Po

we

r (d

Bm

)

Frequency(GHz)

(a)

DAC response

Gaussian response

0 5 10 15 20 25-100

-80

-60

-40

-20

0

Po

we

r (d

Bm

)

(b)

Frequency (GHz)

DAC response

Gaussian response



of 78

Figure 8 (a) SNR estimation and (b) BL considering the high-speed DAC and 20 GHz signal

bandwidth.(c) SNR estimation and (d) BL considering the AWG and 10GHz signal bandwidth.

The laser driving the MZM is modeled as a standard continuous wave laser centered at 1550.12 nm,

with output power 7 dBm and 5 MHz line width, this last modeled as a Wiener process. The optical

SSB filter bandwidth is set to 12.5 GHz or 25 GHz (according to flexi-grid), depending on the

electrical signal bandwidth occupancy. The optical channel is emulated either with a Variable

Optical Attenuator (VOA) or Standard Single Mode Fiber (SSMF). The split-step Fourier method is

used to model the propagation over the SSMF with a dispersion coefficient of 16.5 ps/nm/km, a

nonlinear coefficient of 1.2 W-1km-1 and 0.29 dB/km attenuation. The step length used is set in order

to have a maximum non-linear phase change of 0.5 rad. The receiver is modeled as a PIN

photodiode with 0.7 A/W responsivity, overall thermal noise value of 16 pA/√Hz, and dark current

of 10 nA. At the receiver, two noise contributions have been considered: the thermal noise, modeled

as a Gaussian process, and the shot noise, which is modeled as a Poisson process.

The limiting effect of the high-speed DAC and the AWG has been analyzed and can be observed in

Figure 8 (a) and (c). The reported signal-to-noise (SNR) per OFDM subcarrier has been estimated at

the receiver DSP for the optical back-to-back (B2B) configuration, using uniform loading. High-

frequency subcarriers are attenuated due to the DAC limited bandwidths degrading the overall

system performance. By implementing bit and power loading algorithms as described in the report

of milestone MS6 [MS6], link impairments and hardware limitations, such as the DAC reduced

bandwidth, can be mitigated [Nadal14]. In addition, the system flexibility and scalability can be

(a)

0 20

0 100 200 300 400 5005

10

15

20

25

30Frequency (GHz)

Ncarriers

SN

R (

dB

)

0 100 200 300 400 5000

1

2

3

4

5(b)

Ncarriers

Bit L

oadin

g

0 100 200 300 400 5005

10

15

20

25Frequency (GHz)

Ncarriers

SN

R (

dB

)

(c)

0 10

0 100 200 300 400 5000

1

2

3

4

5(d)

Ncarriers

Bit L

oadin

g



of 78

enhanced. The OFDM subcarriers are loaded with different number of bits and suitable power

values according to the SNR estimation (see Figure 8 (b) and (d)). Rate adaptive algorithms are

studied to maximize the data rate for a fixed performance [Cioffi]. Bit Error Rate (BER) of 3e-3 is

considered in the numerical analysis for a Forward Error Correction (FEC) coding overhead of 7 %

[Nadal14].

Table 1 Achieved numerical results

Scheme Channel DAC BW [GHz] Guard-interval

[GHz]

Data rate

[Gb/s]

BER OSNR

[dB]

SSB B2B w/o 10 10 49.2 2.7e-3 34

SSB B2B High-speed 10 10 46.2 2.7e-3 33.4

SSB 150km SSMF High-speed 10 10 35.1 2e-3 30.7

No guard-

interval SSBB2B w/o 20 0 75.2 2.2e-3 34.6

No guard-

interval SSBB2B High-speed 20 0 64.1 2e-3 30.6

No guard-

interval SSB150km SSMF High-speed 20 0 36.7 2.5e-3 21.4

No guard-

interval SSBB2B w/o 10 0 39.7 2.4e-3 35.1

No guard-

interval SSBB2B AWG 10 0 37.3 2.2e-3 35.1

No guard-

interval SSB150km SSMF AWG 10 0 26.4 2.7e-3 34

In order to maximize the system performance and further enhance the system flexibility, the

adaptive subcarrier loading can be combined with the programmable selection of the optical carrier

by means of the TLS and of the SSB filter center wavelength and appropriate bandwidth.

Furthermore, it is important to point out that the MZM bias point could be also controlled for the

optimization of the optoelectronic front-end [Li13].

Simulations have been performed, comparing SSB and no guard-interval SSB modulation schemes

using a high-speed DAC and with total bandwidth occupancy of 20 GHz (as seen in Figure 6 (a) and

(c)). For the B2B transmission of the SSB-OFDM, 46.2 Gb/s data rate is obtained at a BER of 2.7e-3

with 33.4 dB OSNR. Whereas, a higher data rate of 64.1 Gb/s with 30.6 dB OSNR is achieved at the

target BER, implementing the no guard-interval SSB scheme. We have also analyzed the system

performance after 150km of SSMF, in order to evaluate the resilience of both schemes against fibre

impairments. Optical amplification has been performed every 50 km of SSMF. Figure 9 (a) shows

that the OFDM subcarriers corresponding to high frequencies are attenuated; however, 35.1 Gb/s



of 78

can be transmitted with the SSB scheme at 2e-3 BER with 30.7 dB OSNR. When no guard-interval

SSB configuration is implemented, a similar data rate (36.7 Gb/s) can be obtained at the target BER

with 9.3 dB less OSNR. In this case, the OFDM subcarriers are attenuated at low and high

frequencies (as it can be seen in Figure 9 (b)). Nevertheless, all the available bandwidth, including

the guard interval, is adaptively filled with data, allowing achieving better performance compared to

the SSB scheme.

Hence, no guard-interval SSB outperforms SSB either in the B2B or after 150 km SSMF, arising as

a potential solution for the design of OFDM-based BVT, which can be integrated within a software-

defined sliceable BVT architecture using DD. Consequently, additional analysis has been performed

considering the AWG response and the no guard-interval SSB-OFDM with smaller electrical

bandwidth occupancy of 10 GHz. At 2.2e-3 BER, 37.3 Gb/s transmission is achieved with 35.1 dB

OSNR. The target BER can also be guaranteed after 150 km SSMF, transmitting 26.4 Gb/s data rate

with 34 dB OSNR. The achieved numerical results are summarized in Table 1.

Figure 9 SNR estimation after 150km of SSMF considering (a) SSB and (b) no guard-interval SSB

schemes.

Finally, experimental validation of the no guard-interval SSB DD system of Figure 5 is provided.

The 10GHz bandwidth arbitrary waveform generator (AWG) at 20GSa/s and a real-time

oscilloscope (DPO) working at 100GSa/s are used at the transmitter and receiver side, respectively.

The optical channel is replaced by a VOA. Erbium-Doped Fiber Amplifiers (EDFA), an Optical

Spectrum Analyzer (OSA) and an optical band pass filter of 50 GHz bandwidth are used for OSNR

measurements. Preliminary experimental results of the B2B configuration are shown in Figure 10,

considering a 10 GHz electrical signal. A maximum data rate of 26.8 Gb/s is achieved for 30.4 dB

OSNR for a BER below the threshold of 3e-3. These preliminary results using the AWG have been

included in [Svaluto15]. We have also set-up an OFDM-based BVT using the high-speed DAC (at

64GSa/s) as shown in Figure 10. They have been integrated (in the framework of WP4) in the S-

BVT used in the joint experiment (reported in D4.2).

10 15 20

(a)

0 100 200 300 400 50010

15

20

25

30

Ncarriers

SN

R (

dB

)

Frequency (GHz)

0 10 20

(b)

Frequency (GHz)

0 100 200 300 400 5005

10

15

20

25

Ncarriers

SN

R (

dB

)



of 78

Figure 10 Experimental BER performance in B2B configuration and achieved data rate vs

OSNR (left); experimental set-up using AWG (top right) and using the high-speed DAC

(bottom right).

2.2 DMT transceiver

2.2.1 Comparison of modulation techniques for 400 Gb/s Ethernet transceiver

Figure 11 shows the modulation techniques proposed in IEEE 400 Gb/s Ethernet Task Force

(IEEE802.3bs) [Stassar14]. The proposed techniques are grouped by bit rate per lambda. One group

is 50G/lambda, which needs 8 wavelengths to achieve 400Gb/s. 50Gb/s NRZ and 50Gb/s (25Gbaud)

PAM4 are proposed for this group. The other group is 100G/lambda, which needs 4 wavelengths to

achieve 400Gb/s. 100Gb/s(50Gbaud) PAM-4 and 100Gb/s DMT [Tanaka14] [Isono14] are proposed

for this group. In this section, the modulation techniques are compared from the aspect of bandwidth,

power consumption and cost.

Figure 11 400GbE proposal on IEEE802.3bs

12 14 16 18 20 22 24 26 28 30 320

10

20

30

Da

ta R

ate

(G

b/s

)

OSNR (dB)

12 14 16 18 20 22 24 26 28 30 320.1

3

10x 10

-3

BE

R

Data rate

BER



of 78

Figure 12 shows relation between power penalty of transceiver responsivity and bandwidth of

transmitter and receiver calculated by numerical simulation. Required bandwidth for

56Gbps(28Gbaud) PAM-4 and 116Gbps DMT is almost the same and 14GHz is required to achieve

power penalty of 1dB. This shows efficiency for bandwidth of DMT it twice better than PAM4.

Figure 12 Required bandwidth for each modulation techniques

Figure 13 shows comparison of relative cost for each modulation techniques. Relative cost is

estimated by comparison with the cost of current 100G(4x25G NRZ) transceiver as reference. Cost

of DMT is lower than other techniques because of the smallest quantity of components (4

wavelengths) and the narrowest required bandwidth.

Figure 13 Relative cost comparison for each modulation techniques

Figure 14 shows comparison of power consumption for each modulation techniques. The power

consumption is estimated by components of transceiver, those are optical transmitter (TOSA),

optical receiver(ROSA) and signal processing IC. For DMT, power consumption of TOSA and

ROSA are estimated smaller than other techniques because of the smallest quantity of components

(4 wavelengths) and the narrowest required bandwidth. On the other hand, power consumption of

signal processing IC occupies largest in DMT transceiver. Total power consumption of DMT



of 78

transceiver is smaller than 50Gb/s NRZ or 50Gb/s PAM4, but is larger than 100Gb/s PAM4

transceiver. However, the power consumption of signal processing IC will be decreased by future

CMOS process migration.

Figure 14 Power consumption comparison for each modulation techniques

In summary, DMT transceiver has high efficiency for bandwidth and the smallest quantity of

components with narrowest bandwidth is needed compared with other modulation techniques. As a

result, DMT transceiver can achieve lower cost and power consumption than other modulation

techniques.

2.2.2 Development of 400G(4x100G) DMT transceiver demonstrator

The FPGA-based DMT transceiver demonstrator was revised to support 400G(4x100G)

transmission. DMT modulation and demodulation is performed by FPGA board. HAPS-62 with

Virtex6 was used for FPGA board. DMT transmission of each wavelength was done sequentially.

The setup of transmission experiment of DMT transceiver demonstrator is shown in Figure 15.

Transmission characteristics of demonstrator were verified by back-to-back and 40 km transmission

configuration. Table 2 shows the experimental results. Data rate >112 Gb/s with BER <3e-3 was

achieved for back-to-back and data rate >100 Gb/s with BER <3e-3 was achieved for 40km

transmission for all four channels.



of 78

Figure 15 Transmission experiment setup of 400G(4x100G) DMT transceiver demonstrator

Table 2 Transmission characteristics of 400G(4x100G) DMT transceiver demonstrator

Back-to-Back 40km transmission

Data rate BER Data rate BER

Ch1 117.0 Gbps 2.99x10-3 101.3 Gbps 1.80x10-3

Ch2 120.6 Gbps 2.90x10-3 109.1 Gbps 2.29x10-3

Ch3 112.9 Gbps 2.72x10-3 100.2 Gbps 1.86x10-3

Ch44 112.9 Gbps 2.50x10-3 100.3 Gbps 2.29x10-3

2.3 Spectral-efficiency transmission in EON with high-baud rate multi-

format transceiver

2.3.1 High-baud rate multi-format transceiver

Flexi-grid DWDM technologies provide large-capacity spectral-efficient transmission optical links.

Increasing the spectral efficiency with high order modulation format signals provide a promising

solution to improve the transmission capacity without resorting to higher bandwidth devices.

Nyquist WDM technology adopts pulse shaping technologies to provide Tbit/s transmission using

spectral-efficient optical signals [Zhou12]. Coherent optical orthogonal frequency division

multiplexing (CO-OFDM) technology shows subcarrier flexibility and high spectral efficiency.

Tbit/s optical transmission had been demonstrated over a long transmission distance [Liu12]. Single

Carrier Frequency Division Multiplexing (SC-FDM) signals, being a modified form of CO-OFD,

can inherit the advantages such as low computation complexity and high flexibility, while suffering

less nonlinear impairment due to much lower Peak-to-Average Power Ratio (PAPR) [Kobayashi12-



of 78

1]. WDM transmission of SC-FDM signals had been demonstrated over about 1200 km with a net

spectral efficiency about 8.96 b/s/Hz [Kobayashi12-2]. However, for the above-mentioned

technologies, electrical digital-to-analog converters (DACs) are indispensable components of the

transmitter design. The states of the art DACs show limited electrical bandwidth, comparing to that

of other electrical device. Thus a single transmitter only can operate at a low baud rate, which means

that to realize a high capacity transmission link, several transmitters should be combined together to

form a Superchannel Signal (SC). In the view of practical engineering, each transmitter in a super

channel consists of DACs, drivers and modulators, which make it much expensive than the

traditional single carrier solution.

The single carrier based transmitter can operate at a relative high baud rate with spectral-efficient

high-order modulation format signals. Transmission of 1200-km with 41-Gbd PDM-64QAM signal

was demonstrated using a single I/Q modulator [Peng12]. With multi-formats support, the high baud

rate transmitter can change the spectral efficiency and lead to a capacity variable transmitter. The

multi-format transmitters provide a trade-off between transmission distance and link capacity. Baud

rate adaption is another solution for BVTs. However, this solution provides little margin adaption

[Auge13]. In addition, it’s a waste of hardware resources to operate the transmitter at a low baud

rate compared to its designed optimum operation baud rate. The multi-format transmitters change

optical bandwidth without affecting the occupied optical bandwidth, and provide a big transmission

margin with different modulation formats. Thus for the OCS domain, single carrier based multi-

format transmitter is a promising solution for both network flexibility and high bandwidth in long

distance optical links. So in OCS domain for Strauss project, multi-format single carrier transmitter

provides a good solution for flexible and spectral-efficient transmission.

A QPSK/16QAM multi-format transmitter is setup with baud rate up to 40 Gbaud. The detailed

setup is shown in Figure 16. The OPS-OCS interface card, implemented using high performance

field programmable gate array (FPGA) (HTG Xilinx V6 PCIE board), receive Ethernet traffic using

multiple 10GE SPF+ modules, and then process and groom the traffic to drive a sliceable,

programming BVT.

Figure 16 Experimental setup of QPSK/16QAM multi-format transmitter

The groomed and classified ingress traffic from multiple access links is then assigned to different

egress ports. A 4:1 electrical multiplexer is used to multiplex four data streams from four egress

ports to 40 Gbit/s electrical signals. Then the achieved 40 Gbit/s electrical signals drive an IQ



of 78

modulator to modulate 8 external cavity lasers (ECLs) to obtain 40 Gbaud QPSK signals. The CWs

of 8 ECLs are tuned within a 100 GHz grid for 40Gbaud signals. Then a polarization multiplex (PM)

stage is used to achieve 40 Gbaud PM-QPSK signals. Part of the 40 Gbaud QPSK signals is

launched into a QAM16 emulator (Kylia) to achieve 40 Gbaud 16QAM signals. Again, a PM stage

is used to double the spectral efficiency to achieve 40 Gbaud PM-16QAM signals. Then, the

generated QPSK and 16QAM signals are launched into a 4:1 SSS (Finisar Waveshaper) for future

re-configurability. Figure 17 shows the eye diagrams of the generated 40 Gbaud QPSK and 40

Gbaud 16QAM signals in Figure 17 (a) and Figure 17 (c). The recovered constellation diagrams are

shown in Figure 17 (b) for QPSK signals and in Figure 17 (d) for 16QAM signals.

Figure 17 Eye diagrams of the generated 40Gbaud QPSK signal (a) and 40 Gbaud 16QAM signal (c).

The corresponding recovered constellations are shown in (b) and (d).

2.3.2 Impairment mitigation with receiver-side digital signal processing

Optical signals will degraded due to the fiber impairments, such as fiber dispersion, and polarization

mode dispersion (PMD) during optical transmission. In addition, a typical temperature-stabilized

wavelength locker for Wavelength-Division-Multiplexed (WDM) systems has a wavelength

accuracy of about ±1.25 GHz [Gnauck09]. The Carrier Frequency Offset (CFO) also will affect the

transmission performance.

Thus, in STRAUSS project, we developed a full set of DSP algorithms to handle the transmission

impairment. Figure 18 shows the developed function blocks for offline processing.



of 78

Figure 18 Function blocks for offline DSP processing

The offline DSP algorithm handles fiber dispersion (CD Compensation), PMD (Polarization

Demultiplex) and CFO (FOC). Figure 19 shows the developed GUI for DSP algorithms.

Figure 19: Control GUI for offline digital signal processing

In high baud rate transmitters, the CFO would affect the transmission performance dramatically due

to the required large capacity. In our DSP algorithm, we choose a spectrum based frequency offset

estimator [Gnauck10] to make it suitable for 16-QAM systems.



of 78

2.4 Real-time OFDM Transmitter with Ethernet MAC

The project considers also real-time implementation of critical subsystems to ensure that industry

can implement the investigated schemes finally using state-of-the-art analogue and digital

components. The real-time OFDM transmitter is able to offer service up to 1024 slices in parallel

and enables the use of bit and power loading. The highest supported modulation format is 16-QAM.

That enables a maximal achievable gross data rate of 64 Gbit/s at a sample rate of 16 GS/s as

already described in detail in deliverable D2.1. A short overview is given below followed by the

second part describing the Real-time MAC implementation.

2.4.1 Real-time implementation

For the realization of the real-time OFDM transmitter a single Virtex 6 XC6VHX565T was used to

generate the in-phase (I) and quadrature (Q) signal for the two high-speed MICRAM DACs. Figure

20 illustrates the implemented functionality in the FPGA-based transmitter. The frame generator

block generates random payload data for the OFDM transmitter preceded by a header containing bit

loading information. Moreover, the frame generator easily allows the insertion of user specific pilot

tones. The bit loading information is firstly extracted and stored, and later it’s applied on the

following random data at the mapping block. The supported modulation formats are BPSK, QPSK

and 16-QAM. The subcarriers can also be left unmodulated by setting the signal to zero. A training

sequence of 20 OFDM symbols is sent periodically every 192 symbols for frame synchronization

and channel estimation at the receiver. Power loading is done on all subcarriers before the 1024-

point IFFT to adapt the subcarriers amplitude. This stage consists of multiplying the subcarriers by

real-valued power coefficients. After the 1024-point IFFT, the complex OFDM samples are scaled

and clipped to be 5 bits width, both real and imaginary parts, due to the limited numbers of serial

IOs. Then a cyclic prefix of 64 OFDM samples is inserted at the beginning of every OFDM symbol.

Finally the data structure is adapted to the format required by the DAC interfaces which expect 128

parallel complex samples.

Figure 20: Implemented functionality in the FPGA-based real-time OFDM transmitter



of 78

FPGA-based signal processing is an attractive option for real-time signal processing implementation

in networks. However, despite its clock speed of hundreds MHz, data throughputs of several tens

Gbit/s can be achieved by parallel signal processing. For a given target throughput, the required

clock speed decreases proportionally to the number of samples processed in parallel. At the same

time, the FPGA resource consumption increases proportionally to the parallelism grade. Both high

clock speed (≥250 MHz) and high FPGA resource consumption turn real-time implementation into a

challenging task. The networks targets translate into ambitious design requirements. Resource-

consuming implementation is needed due to the IFFT size and high data throughputs to be processed

for the high aggregate data rate. From the hardware development point of view and in order to

minimize the complexity, a tradeoff between feasible clock speed and limited resource consumption

has to be made.

In such a high performance application pipelined architectures are of vital importance to meet the

FPGA timing requirements (clock speed). Since pipelining consists of adding intermediate buffer

stages to break combinatorial paths and reduce processing delays (increase clock speed), it also will

impact on the overall resource consumption. The likely congested resulting design will need

optimization for a successful and efficient implementation. This consists of proper coding, careful

description of timing requirements that will be passed to the design software as constraints, as well

as manually placing blocks of logic in the FPGA.

The real-time OFDM transmitter presented Figure 21 consumes 27% of the available slice registers,

38% of the available LUTs (Look-Up-Table), 72% of the available DSP slices and 100% of the

available GTXs.

Figure 21: Top view on the FPGA board and connection to the DAC

2.4.2 Real-time MAC Layer design

The MAC Layer design also was realized with FPGA technology. The demonstrated data rate is 10

Gbit/s.

The system offers the possibility to realize OFDMA, which is a point-to-multipoint connection in

conjunction with an appropriate switching to operate in OFDM technology. The solution is prepared

for that and it takes only a few amendments enabling this functionality.



of 78

The system is largely flexible with respect to input and output data rate, modulation format and rate

adaptation. With the FPGA chip of the selected platform (Xilinx Virtex-6, XC6VHX565T) the

following parameters can be achieved.

Up to 512 Carrier

40 Gbit/s data rate

The MAC layer also offers possibility for higher modulation formats. Ignoring the limitations of the

physical layers hardware, 256-QAM format is possible in the actual design.

The description below is based on the low demand values at the outset of this clause. At appropriate

parts of the description, it is outlined how to reach higher values.

For the demonstrator, transmitters and receivers have been implemented on the same hardware in

order to make optimal use of existing resources. However, they operate independently, so both

components can easily be implemented locally separated using additional hardware.

Figure 22: MAC to PHY FPGA realization



of 78

The MAC layers system boundary is given by the FPGA development board containing the above-

mentioned Virtex-6 chip from Xilinx. FPGA chips offer flexible interconnections, parallel logic

consisting of LUT-flip-flop combinations and tailored functional blocks, such as e.g. memory and

transceiver blocks.

The external interface for WAN and Upstream interconnection is realized with optical 10 Gbit/s

Ethernet interfaces standardized by IEEE802.3ae. The interface to the physical layer uses parts of

this specification, but saves the Ethernet MAC layer due to lack of a fitting higher layer format.

Physically it is implemented on four parallel, differential, electric lines. However, almost any other

interface will fit as long as it offers continuous transmission and at least enough bandwidth for the

incoming data. On the input side is a higher bandwidth possible by adding additional parallel 10

Gbit/s Ethernet data lines.

To make the high gross clock rates of 10.3125 GHz (10 Gbit/s Ethernet IEEE 802.3ae including 64B

/66B line coding) processable within an FPGA an internal parallelization of these serial data streams

is necessary.

Due to the flexible interconnect structures within the FPGA a maximum processing speed of about

of 250 MHz is possible. The needed parallelism is already done by the chip’s high-speed

transceivers having internal 64 bit interfaces. Regarding the automatically decoded line coding it

results in an easy manageable clock rate of 156.25 MHz. Other components of the development

boards are merely for optoelectric conversion, data regeneration and clocking.

The following figure shows an overview of the subsystems components:

Figure 23: Sender design

Figure 23 shows no detailed description for the upstream path. The upstream isn’t provided for

OFDM transmission yet. An upstream has to be provided anyway to get duplex transmission

possibilities needed by most transmission use cases. It is realized as a non-converted 10 GBit/s

Ethernet path, routed through the sender component to use common duplex on the downstream data

providing SFP+-site and to obtain access to the content of the back path.



of 78

The downstream is divided into three blocks:

Cutthrough: The data will be analyzed for further processing.

MAC functionality: Includes destination addressing for parallel subcarrier data streams,

caching of these data streams and adding FEC data.

Format and rate adaptation: The parallel data streams are adapted to the transmission format

of 64 parallel bits needed by the transceivers and to the number of bits corresponding to the

current valid modulation format of the Physical Layer.

Outside of these blocks are the transceivers and various IP cores managing applied transmission

formats:

PCS / PMA: configuration and control of the GTH transceiver enabling there single line

Ethernet capabilities

XAUI: Configuration, control and synchronization of the GTX transceivers using four

parallel lines for an Ethernet similar format

Ethernet MAC: extraction of the Ethernet layer-2-frame data from the data stream as well as

handling of Ethernet preamble and FCS

Blocks for minor functionality (e.g. to configure SFP+ modules) as well as some additional signals

(e.g. flow control between MAC and PHY layer) are not shown for the reason to concentrate on the

major functions.

The three background colors represent clock domains valid for data processing in the

subcomponents. They are provided by the transceivers based on their reference clock corresponding

to their internal 64 bit interface. It is possible to use just one clock in all data paths since all

transceivers use the same clock rate (ignoring lowering the net data rate by adding about 20% of

FEC overhead) and clock phase is aligned by the controlling IP cores. To keep flexibility to enhance

clock rates compensating the additional overhead, the design is divided in these three clock domains,

interfacing them by memory block needed anyway natively bringing the possibility to use different

clock domains for reading and writing

Below the three downstream blocks are described in detail:

1) Ethernet-Cutthrough

For forwarding the incoming Ethernet data extracted Ethernet MAC data (Ethernet frame) has to be

analyzed in order to allow the proper processing.



of 78

Figure 24: Ethernet Frame

For the desired functionality, it is necessary to know the length of an Ethernet frame, which must be

reconstructed on the receiver side. The receiver needs length information before receiving the frame

to determine the end. To use the low latency “Cutthrough” approach the length information is read

out from data stream. A more simple way was just counting the incoming 64 bits (8 bytes) words.

The alternative idea of using an end tag was not followed, because frame data could take any form.

For this approach an end tag has to be extremely long not to be accidentally mistaken with payload

data and should take changing forms not to be generated by simple data injection.

Figure 24 shows the format of an Ethernet frame. The length itself can be found in the third 8 byte

word if no higher layer protocol is used. In most cases, a higher layer protocol is encapsulated,

which is defined by a type information in length field. The content length is fixed (e.g. ARP) or can

be found at a fixed position the beginning of higher layer data (e.g. TCP/IP). Normally it is not

necessary to read more than four words to get this information.

In all cases, destination address has been passed (located at the second 8 byte word). Extracting it

has the advantage to prepare the system for OFDMA functionality where Ethernet frames have to be

switched to different destinations.

Figure 25: Ethernet-Cutthrough-Component

The described functionalities are realized as shown in Figure 25. The incoming frames are written

into a FIFO memory. The arrival is detected by an accompanying “valid” signal (a “valid” bit

depending per incoming byte, so 64 data and 8 valid bits are on the interface). In addition to storing



of 78

the first data word of a frame they are analyzed by a state machine, so that Ethernet destination

address (destination MAC address) and length information can be extracted.

If this information is known, Ethernet destination address is passed to an Ethernet MAC lookup

table (MAC-LUT) to get the corresponding destination for OFDMA. For a pure OFDM system, the

functionality is bridged by a constant MAC-LUT answer, so all data is passed to the same and only

destination. Beside, all incoming data is continuously saved to the data FIFO.

If the length and real or generated destination information are available, the length information may

be placed at the beginning of the saved frame in an appropriate manner and the data can be further

processed. Thus several buffered frames can be processed at the same time. Extracted information is

written in a parallel FIFO memory. The simultaneous backup of several frames and related

information is necessary for short frames, when the MAC-LUT (if any) is already busy or fetching

data from data-FIFO is delayed by a following component.

For providing the length information at the beginning of the frames at the receiver is written in the

end of the preamble, which has no technical significance for today’s Ethernet systems.

The preamble is used at the receiver in a second way. The alternating “10” bit sequence (the original

form of the preamble that is kept in the beginning) is used to detect the start of a frame. Since just

one of the original 7 bytes are needed for it, more system information can be added to enhance

functionality, for example a sequence number to be able to correct reordering arisen by parallel

frame transmission on different speeded OFDM sub carriers.

For message passing to the following subcomponent a state machine was implemented. It signalizes

a new analyzed frame and awaits an answer. If the frame is declined (“reject”), it is thrown away by

just reading the FIFO without forwarding. The normal case is a positive answer (“accept”). Then it

passes the modified preamble, followed by the original frame data. To avoid blocking, the

Cutthrough component throws away a frame by itself when it gets no answer in a certain time

window. But, if all values are set proper, the “reject”-signal should appear earlier.

To enhance input bandwidth, there are several ways. The system is prepared for handling multiple

10 GBit/s Ethernet inputs by adding additional paths implementing one Cutthrough component each.

Additional possibilities may be given by higher parallelism (bigger than 64 bit) in an aggregated link

or higher speed transceivers (as available on latest Virtex FPGA devices). The advantage is more

simple connectivity (single link) and a simpler routing by the following component. The

disadvantage is higher resource consumption in the FPGA by wide busses, so that fewer sub carriers

can be realized by comparable devices.

2) The MAC-function block

The MAC function block (see in Figure 23), consisting of the switching-, the buffering-, and the

FEC-subblock manages the main task of the OFDM sender: Distributing the analyzed, prepared, and

asynchrony incoming data to sub carrier data streams.



of 78

Figure 26: MAC-Function Block

Figure 26 shows more details. On the most left side, one can find the cut-through block again to

clarify the interface partners. It communicates with the switching block. In this figure the data path

(“MUX”) has been extracted from the decision path (still named “switching”). The data paths MUX

is switched by the decision and forwards the data to the carrier buffers. The decision will be made as

follows:

The Cutthrough path signaled an analyzed and reformatted frame. With OFDMA

functionality, the signal is accompanied by the client number, with pure OFDM

functionality, this information is constant.

The decision unit checks the fill level of following data buffers and can thus detect which

buffer has sufficient capacity. Because of the low subcarrier count in the described system

the system resulting in low buffer number the buffers were chosen big enough to

accommodate multiple Ethernet frames of maximum size. So the threshold level (“full”) is

set to a fixed value. For a significantly higher number of carriers it is helpful to evaluate the

size of the frames, to limit the buffer size and to make a more accurate evaluation of the fill

level. Following this approach limited FPGA resources are utilized better.

For OFDMA another part is in the decision logic, which makes it possible to apply certain

subcarriers to a particular client. The assignment can be done at system start or dynamically

at runtime. It will be incorporated to the decision. Also conceivable is an internal loop

provided with additional logic that evaluates “full and empty statistics” and this way decides

to apply carriers dynamically to client destination. For pure OFDM all carriers are assigned

fix to the only client fixed before startup.

If a positive decision has been made, the multiplex path is switched accordingly and the

Cutthrough component is instructed to begin the data transfer. If all buffers are full, the

Cutthrough logic is instructed to discard the frame.

The data FIFOs provide the interface between the receiving and the transmitting clock domain. They

are implemented in FPGA memory blocks bringing the possibility of using two different clock

domains on read and write port natively. Additional resource usage is very little for this

functionality.



of 78

The memory FIFOs are read out continuously through the Reed-Solomon-FEC encoders. The

encoders add overhead data allowing the receiver to reconstruct data harmed by transmission errors.

Each subcarrier has got one encoder working independently. This way different bit widths by

different modulation formats can be handled not mixing up the payload-overhead-sequence of the

decoders. For OFDMA the second reason is, that different subcarriers reach different destinations.

The forward error correction (FEC) part is divided in two parts: The encoder itself and the serializer

allowing splitting the encoders fixed word width to the dynamic bit width given by physical layers

actual modulation format. The requests are done by the “Format- and Rate-Adaption-Unit”

following up in the data path.

Data request is done without knowing about the fill state of the corresponding buffers. Underrun is

allowed and even brings an additional function: Since each frame written in the buffer will close

with an idle word (“0000 0111”) and buffer underrun results in stay of the last valid word, it’s for

sure no frame start is detected accidentally by the receiver in contrast to sending random data or the

last valid data word. The chosen sequence brings just single “1-0” and “0-1” bit changes resulting in

not being similar to a shortened preamble as used for frame start detection in the system. Constant

words will bring the same feature, but result in constant FEC-overhead-blocks and in an unfavorable

spectral signal.

3) Format- and Rate adaption

For transmission to the physical layer, a type conversion must take place. In addition, this

component receives currently valid bit loading information, which is determined by the current

subcarriers modulation format.

In most cases the OFDM symbols bit width doesn’t match the interface width. Therefore it’s

necessary to adapt the format by distributing a symbol on more than one transmission word. The

example case brings 96 bits per symbol but 64 bits on the XGMII-interface borrowed from the 10-

G-Ethernet specification. Further on there may be additional demands by the interface, like XGMII-

transmission-starts may only be at the first or the fourth byte.

For synchronization reasons between MAC and physical layer, transmission is stopped for a few

clock cycles by a regular rhythm. If the input buffer of the physical layer device is going to overflow

soon, a longer transmission pause is requested. The transmission-to-pause-ratio is chosen in a way

that resulting data rate is just a bit higher than the physical layer is transmitting, respectively it’s just

a bit slower when requesting a longer pause. The data transmission cycle is constant and has a length

corresponding to multiple FEC-block-lengths. So, if a transmission period and a FEC-block start

together, each following transmission block starts again aligned with an FEC-block as long as the

length is an integer divider of all allowed bit width and FEC-block-length. For this reason, change of

the modulation format (resulting in a different bit width) is only allowed during the short

transmission pause to keep alignment. Alignment to modulation format change ensures not to lose

bits in the physical layer and alignment to FEC-blocks ensure having a synchronization point at the

receivers MAC layer if the interface signal is reconstructed the same way.



of 78

To keep constant interface words, non-used bits by low modulation formats are padded and ignored

by the physical layer.

Rate adaption is applied externally in the demonstrated system. Therefore a low bandwidth SPI-

Interface is used. Rate adaption also could be done by the receivers MAC layer by evaluating the

number of corrected errors of the Reed-Solomon-FEC-encoder. To transmit the results to the sender,

a parallel channel (like SPI) or an inband channel in the upstream could be used. All participating

devices (physical- and MAC-layer, receiver and sender) have to be informed about a change, react

in the processing order and signalize a change done (or keep a fixed timing).

3 Flexible/adaptable optical nodes for flexi-grid DWDM networks

To handle the heterogeneous and dynamic Ethernet traffic in flexi-grid DWDM networks,

Architecture-on-Demand (AoD) based flexible modular and scalable optical node is proposed using

a large-port-count fiber switch (referred as AoD switch). Figure 27 shows the design of the proposed

AoD-based flexible modular and scalable optical node.

Figure 27: Design of AoD-based flexible modular and scalable optical node

The proposed optical node comprises of four main parts. A Large Port count Fiber Switch

(LPFS), used as an optical backplane, provides a large scale switch matrix and enables the

programmable feature of the proposed optical node. Several technologies, such as 3D MEMS

(Micro-Electro-Mechanical System) and beam steering technology, provide optical fiber switch

matrix up to 320×320. The available LPFS enable the synthesis of large scale AoD optical node with

the subsystem & component inventory. The inventory as another key parts of the node design shown

in Figure 27, is composed of several stand-alone subsystems to provide network functions. The

subsystem inventory comprises of superchannel add/drop, fast time switching for Time Division

Multiplexing (TDM), OPS/OCS interface, et al. A lot of optical common components, such as

optical amplifiers (EDFAs), optical couplers, optical splitters, and Spectrum Selective Switches

(SSSs), are available in the inventory to provide essential function for optical node function



of 78

synthesis. To enable the programming and reconfiguration, all the input and output ports of these

function modules or subsystems will be connected to the LPFS-based optical backplane. Thus, the

LPFS-based optical backplane can manage interconnections between the function modules and also

handles both the input and output ports of the optical node.

Another part of the designed optical node is the node composing algorithms. The node composing

module talks to the control layer and response the network requests, including node functions,

wavelength allocation, and bandwidth requirements, then synthesis optical node architecture by

configure the large port count fiber switch based optical backplane. Some node composing

algorithms will be deployed here to balance the payload of the optical node to use the network

hardware efficiently. The optical node synthesis would reduce the total energy consumption, as the

unused device can be turn down when Ethernet traffic is low. The AoD-based optical node provides

an efficient way to allocate optical hardware resources.

Optical monitoring block is the third part of the AoD-based optical node. The inline power monitors

are integrated at both the input and output port of the LPFS. Compared to the traditional power

monitoring, the ubiquitous inline power monitors enable the monitoring module monitor all the

components and subsystem at same time. The AoD-based architecture enables optical characteristics

of optical components and subsystems by insertion loss and gain. As optical power reflects

important information about optical signals, e.g., optical power indicates spectral usage in optical

links. The power monitor block provides a lot of information about the optical node. The monitoring

information will be feedback to the node composing module for network performance optimization.

The monitor information can also be sent to the control layer for network optimization.

To support the network functions in our proposed AoD-based optical nodes, several network

functions are developed to satisfy the Ethernet traffic requirements. In section

3.1 Development of key subsystem

3.1.1 Software-defined programmable superchannel transceivers

SC signals group several carriers to provide a large capacity transmission. The superchannel

transmitter would satisfy the huge bandwidth requirement of the aggregated data from other network

domains, e.g., the OPS network. With the FPGA-based real-time OPS/OCS integrated interface

developed in Task 2.3.2, the superchannel transceivers can aggregate the Ethernet traffic from OPS

network to a superchannel signal for spectral-efficient transmission. The superchannel transceiver

also provides the modulation programmable feature to provide a large range bandwidth tunability.

Figure 28 shows the proposed design of ingress Ethernet to sliceable SC transponder. The design

consists of the real-time OPS/OCS integrated interfaces (abbreviated as ETH-EON interface) and

optical SC processor that can support assembly, slicing and routing in space and frequency

dimensions. The ETH-EON interface converts Ethernet signals to optical signals with different

modulation formats and different baud rates. The SC optical processor groups any number of

carriers to generate superchannel signals for elastic and high capacity Ethernet transmission.



of 78

Figure 28 Design of Ethernet to software-defined sliceable superchannel transponder

Section 5 shows the detailed design of the real time ETH-EON interface, which is implemented

using high performance FPGA-based optoelectronics (HTG Xilinx V6 PCIE board) with multi-

format transmitters. Incoming Ethernet traffic is received using multiple SFP+ interfaces. Then the

ingress traffic is aggregated and groomed based on its destination MAC address and the virtual

network slices they belong to. The ETH-EON interface can handle variable size of Ethernet traffic

(with VLAN tag) from 64 Bytes up to 1522 Bytes and flew-in on different bit rates up to 10Gbit/s.

The output ports use differential SMA interface.

At the output stage of the FPGA-based ETH-EON interface, the aggregated n data streams are

separated into groups to drive several optical transmitters for further aggregation in the EON

domain. The multi-format optical transmitters, connected to different output port, generate optical

signals with different modulation formats or different symbol rate. By sending the groomed traffic to

different output ports, the E-EON interface can adopt different modulation formats or baud rate to

optimize the transmission performance. The variable baud rate, multi-format transmitters provide

different link capacities for each carrier. The E-EON realizes Ethernet aggregations both in the

electrical domain and in the optical domain.

In order to aggregate Ethernet traffic further, a SC optical processor (add-drop programmable

network of a ROADM) is followed to generate SC signals from several ETH-EON interfaces. A

Polatis fiber switch is used to manage the interconnections of the key components in the SC optical

processor, including SSSs, couplers, splitter and EDFAs, to realize the AoD structure. With the

programmable SSS, the AoD-based SC optical processor can generate different SC signals

according to different application scenarios. The SC signals can provide variable capacity over

Tbit/s for high capacity transmission.

The experimental setup of the Ethernet to sliceable SC transponder is shown in Figure 29. The

sliceable SC transponder provides two SC signals for different application scenarios. SC1 provide a

high-capacity solution with subcarrier modulation format programmability. SC2 signals use low

baud rate QPSK signals target for long transmission links.



of 78

Figure 29 Experimental setup of Ethernet to superchannel converter based on ETH-EON interface and

SC optical processor.

As shown in Figure 29, An Ethernet traffic generator is used to generate Ethernet traffic, which is

fed into the developed Ethernet to software-defined sliceable SC converter with SFP+ interface. The

FPGA-based ETH-EON interface parses the Ethernet frames, and aggregates the traffic based on the

Ethernet header information. According to the adopted strategy, the groomed ingress traffic from

multiple client links is then assigned to different output ports. A 4:1 multiplexer is used to multiplex

four data streams from four FPGA ports to 40 Gbit/s electrical signals. Then the achieved 40 Gbit/s

electrical signals drive an IQ modulator to modulate 8 ECLs to obtain 8 channel 40Gbaud QPSK

signals. The 8 ECLs have high output powers about 15 dBm and low linewidths less than 100 kHz

(Yenista OSICS TLS-AG WDM tunable laser source), which are capable to support high

modulation formats. The 8 carriers are tuned within a 100GHz grid for 40Gbaud signal. Then a

polarization multiplex (PM) stage is used to achieve 40Gbaud PM-QPSK signal. Part of the 40

Gbaud QPSK signal is launched into an all-optical 16QAM emulator (Kylia) to achieve 40Gbaud

16QAM signal. Again, a PM stage is used to double the spectral efficiency to achieve 40Gbaud PM-

16QAM signal. Figure 30 shows the eye diagram of the generated 40Gbaud QPSK (a) and 16QAM

signals (b).

Then, the generated QPSK and 16QAM signals are launched into a 4:1 SSS via a 192×192 fibre

switch (Series 6000, Polatis) for path and function re-configurability. By choosing either PM-QPSK

modulation format or PM-16QAM modulation format, an 8-subcarrier SC signals are obtained with

modulation format re-configurability for each subcarrier. Figure 31 shows the spectrum of the

generated 8-subcarrier SC signals with 16QAM modulation in ch3, 5, 7, and QPSK modulation in

other subcarriers (labelled as SC1). For all the subcarrier, almost error-free back-to-back

(a) (b)

Figure 30 Eye diagram of generated (a) 40Gbaud QPSK signals and (b) 40Gbaud 16QAM signals.



of 78

transmission is achieved. The recovered constellation diagrams are shown as insets in Figure 31. By

adopting different modulation formats and different quantity of subcarriers, we obtained SC signals

with capacities from 160 Gbit/s up to 2.56 Tbit/s.

Figure 31 Optical spectrum of the two SC signals: SC1: reconfigurable SC signals (ch3, 5, 7: 16QAM;

other: QPSK) in 100GHz Grid; SC2: SC signals with QPSK in 20GHz grid.

3.1.2 Programmable ROADM

Another key function for our flexible/adaptable optical nodes is the flexible reconfigurable optical

add/drop multiplexer (ROADM). Colorless Directionless Contentionless (CD/C) ROADM

architectures, which offer additional flexibility and operational simplicity, have generated

considerable interest in optical transport networks. In addition to the CD/C feature, flexi-gird or

gridless is another top-required feature for future ROADM design in elastic optical networks. With

the development of optical networks, large optical bandwidth optical links between two optical

nodes may occupy several wavelength bands or even a full spectrum of a single fiber. The optical

superchannel signals or the full-spectrum signals rise new challenge to current ROADM design. The

full-spectrum signals require optical node can bypass the signals without add/drop operations.

However, the traditional static ROADM design couldn’t offer the bypass function.



of 78

Figure 32 Design of flexible ROADM

Figure 32 shows the proposed design for flexible ROADM. The input and output ports of the key

components in ROADM are all connected to a LPFS. The LPFS synthesis the ROADM functions by

connecting different port together. With different optical components, the ROADM can be

synthesized with different architecture, such as “Broad and Select”, “Route and Select”. Figure 32

shows a synthesized ROADM with “Route and Select” structure. In addition, the used LPFS can

bypass the ROADM to provide fiber to fiber connection, which is very important for future large

capacity transmission.

3.2 Cost and energy consumption analysis

AoD-based optical node provides network functions when needed. Such design will benefit network

hardware utilization efficiency in multi-dimensional optical networks (SDM/WDM/TDM). In multi-

dimensional networks, optical switching in time, frequency, and space need to deploy for all the

fibers to support flexible optical node function. However, it’s impossible and impractical to deploy

all these switching technologies to all the optical paths. The static optical node structure in current

commercialized optical node design couldn’t be extended simply to handle multidimensional

switching operation. On the other side, not all the fiber or optical path need the multi-granularity

switching. Extending optical node switching capability to all the fibers and all the paths will be a

waste of network resources.

Thus the AoD-based optical node design just deploys optical switching functions when needed,

which will lead to a dramatically improvement in hardware utilization efficiency. Even the used

LPFS will add extra cost to optical node design, the LPFS enable new flexibility to provide fiber to

fiber switching, and also enable network function programmability for better network resource



of 78

utilization. To shed a light on cost consumption, some simulated work have been carried for the

AoD-based ROADM.

Table 3 lists the cost in arbitrary units for different devices, considering as a reference the price of a

40-channel AWG-based (DE)MUX. All the information get from different manufacturers.

Table 3 Cost of used components in ROADM

We looked into different ROADM architectures in ref. [Ji10], which listed as No.1 to No.4. The

C/D/C-ROADM architecture No.5 is reported in ref. [Gringeri10]. Our proposed AoD-based

ROADM, listed as No.6, is also evaluated for comparison. As AoD-based ROADM is synthesized

based on the traffic requests, we analysis several traffic requests. Table 4 shows the required

numbers of optical components for different ROADM architectures.

Table 4 Number of devices for different architectures

Active Components Passive Components

Architecture SSS 3D-MEMSa Switchb (DE)MUX Splitter

No. 1 [Ji10] 0 1×{3NW} 0 2N 0

No. 2 [Ji10] N 0 NW/2 0 2N

No. 3 [Ji10] N 1×{NW} 0 2N N

No. 4 [Ji10] 2N 2×{NW} 0 0 3N

No. 5

[Gringeri10]

N 0 NW/2 0 2N

No. 6 N 2×{NW} 0 N N

aNumbers of slow switches ×{size}.

bNumber of fast swtiches 1×N



of 78

The flexibility and reconfigurability of the proposed AoD-based optical node design enable the

optical nodes deploy switch operations where and when they are required. So it’s possible to save

both cost and power consumption for the complex traffic requests in multi-dimensional optical

networks.

By combing Table 3 and Table 4, the total cost can be analyzed. Figure 33 shows costs for ROADM

with different architectures. The AoD-based ROADM show a reduced cost for different node

degrees. When the node degree increases, hardware efficiently will increase and lead to a better

hardware utilization. Currently our preliminary research didn’t consider the future multi-

dimensional optical networks. Actually, the multi-dimensional network benefits more from the

AoD-based node architecture. We will continue work on such topic in the incoming year.

Figure 33 Cost comparison between AoD and different ROADM architectures reported in the

literature.

With respect to energy consumption, we adopt a realistic power model described by the parameters

in Table 6. As AoD-based architecture depends on the traffic pattern, we generate traffic request sets

with the following four parameters:

1) The port load P∈[0, 1] is the fraction of the requested fixed-grid wavelengths per input over W

(i.e., 4 ×12.5 GHz bandwidth channels).

2) The subwavelength index σ∈ [0, P] is the fraction over W of the requested wavelengths per input

that contain TDM subwavelength signals. We consider only two possible destinations for each TDM

subwavelength signal, and thus we focus on the use of 2 × 2 PLZT switches.

3) The superchannel index ρ∈ [0, P] is the fraction over W of requested wavelengths per input port

that are randomly aggregated into couples of two adjacent wavelengths (i.e., 8 × 12.5 GHz

bandwidth channels).



of 78

4) The fiber switch index F∈[0, 1] is the proportion over N of input ports assumed to request fiber

switching in the request set. Therefore, all wavelength and superchannel channels of the input are

switched to the same output. Conversely, the destination of channels that are not assumed to be fiber

switching is set according to a uniform distribution between the output ports.

Different traffic setups (A, B, C, D, E, F) are explored to reflect all the possible traffic situation in

optical node, shown in Table 5.

Table 5 Traffic parameters for each setup

In particular, setup A explores the power consumption of AoD for a fixed-grid wavelength traffic

load of 80%. Setups B, C, and D explore the power consumption of AoD for a traffic load of 100%

with different fractions of subwavelength, superchannel, and fiber requests, respectively. Finally,

setups E and F explore the power consumption of AoD for a traffic load of 100% and with

aggregation into superchannel and fiber requests.

Table 6 Power consumption for optical components

Device Power (W)

Common equipment 100

SSS 40

Fast switch (PLZT) 8

3D MEMS (330 ports) 150

Figure 34 compares power consumptions for the different architectures in Table 4 and AoD under

the traffic setups. All the considered setups for AoD require a lower power consumption than the

architectures proposed in the literature. Furthermore, time switching can be supported in AoD,

unlike other architectures. However, when such capability is exploited, a higher power consumption

is experienced, as shown by setup C. Notably, the inherent flexibility of AoD permits us to save

power compared to other architectures. Power savings depend on the aggregation both into fiber

switching and into superchannels. For instance, a more than 60% power consumption reduction is

achieved in setup B due to traffic aggregation into fiber switching compared to architecture no. 3.

However, as shown in setup D, only 20% of the power consumption reduction is obtained for

aggregation into superchannels only. Higher power consumption savings are obtained when

aggregation into fiber switching and into superchannels are combined (i.e., 50% and 75% of power



of 78

consumption reduction for setups E and F, respectively, depicted with dashed lines). The ability to

switch at different levels (fiber, flex-grid superchannel, and fixed-grid wavelength) reduces the

number of required SSS modules and optical switches, saving on their associated power

consumption. Therefore, power consumption savings are obtained for AoD depending on the traffic

supported (except when time-sliced subwavelengths are supported), thanks to the adaptable nature

of the architecture. Note that such adaptation to the traffic is obviously not possible with hard-wired

ROADMs. Finally, it is worth mentioning that configuration times for AoD and ROADMs are

similar due to the use of subsystems with similar configuration times.

Figure 34 Power consumption comparison between AoD and different ROADM architectures reported

in the literature.

4 Variable capacity OPS based on sliceable transceivers

Optical Packet Switching network is a potential solution for access, aggregation, and/or intra-

datacenter networks as an alternative to energy-inefficient high-speed electrical packet switching

networks. In the previous works on OPS, on-off keying (OOK) format is commonly employed as an

optical payload format [Takahashi14] and thus the packet length is proportional to its capacity

according to the employed baudrate (bitrate). This behavior, although simply extended in electronic

networks, is often inconvenient in OPS networks due to the lack of viable optical RAM buffers or

the difficulty in the optical fiber delay-line (FDL) buffer scheduling.

Meanwhile, recently, multi-level optical modulation formats, mostly developed for optical core

transport network, have been introduced to OPS networks for higher spectrum efficiency. For

instance, DQPSK packet switching has been demonstrated in [Shinada11], 16QAM packet in

[Shinada14], and adaptively-modulated OFDM packet in [Yoshida13]. These advanced modulation

techniques, in fact, are largely owing to digital signal processing techniques followed by high-speed



of 78

digital-to-analog converters, and hence they can be programmable. The flexibility of the packet

modulation format offers a new axis to control the packet payload capacities other than the packet

length. For instance, by changing the modulation format packet-by-packet depending on the required

payload capacity, one can generate optical packets which have a fixed-length, but have variable

capacities to ease the optical buffer scheduling.

In the STRAUSS project, we investigate the OPS networks based on the fixed-length variable-

capacity (FL-VC) optical packet formats. Note that, the network performance may be further

improved by employing a sophisticated variable-length and variable-capacity payload concept.

However, the variable-length, hence asynchronous, OPS networks themselves are state-of-art

technology even with the OOK format at this moment and the further payload format adaptation

both in the time (length) and the frequency (bandwidth) domain there will make the entire network

complexity impractical. The use of fixed-length packets not only eases the optical FDL buffer

implementation, but also eases the adaptive modulation algorithm for the FL-VC packet

optimization. Moreover, in theory, the FL-VC packets discussed here can be transferred over most

OPS network architectures previously developed, e.g., slotted (synchronous), or un-slotted with a

fixed-length or a variable-length, as long as format-free optical switches are employed.

4.1 Single-carrier payload packet versus Multi-carrier payload packet

DSP-enabled advanced optical modulation formats can be classified into two groups, i.e., single-

carrier modulation schemes, such as PSK and QAM, and multi-carrier modulation schemes, such as

OFDM and DMT. Either of the modulation schemes enables the FL-VC concept, i.e., the adaptive

modulation can be implemented either in the time domain or in the frequency domain. (Figure 35)

Figure 35 Images of FL-VC packet based on multi-carrier and single-carrier modulations

In [Yoshida13], we have compared the performance of the coherent optical (CO-) OFDM payload

packet and the coherent signal-carrier payload packet experimentally. Figure 36 show the

exprimental setup for the 1×2 switching and reordering of the OFDM-based and SC-based FL-VC

packets at up-to 30 Gbps. The bit-error-rate (BER) performances versus (payload) OSNR of the FL-

VC packets of 485.5-ns long with different paylaod capacities (9600-14880 bit) are reprenseted in

Figure 36. In the SC-based FL-VC packet, the BER depends on the employed modulation format

rather than the capacity. Meanwhile, we can see smooth relation between the BERs and the

capacities in the OFDM case. In the OFDM-based packets, the available signal energy is efficiently

spread over subcarriers. As a result, the notable improvement can be found for the moderate

capacity (9400-12400bits) packets where no adequate formats are exist for the SC-based packet.



of 78

In the FL-VC packet generation, the subcarrier-by-subcarrier adaptive modulation in the multi-

carrier modulation format offers the finer granularity in the payload capacity comapred with the SC

modulation format, and it can contribute to improve the optical energy efficiency of the payload

generation.

Figure 36 Experimental setup and optical waveform at each observation point ( (a) to (d))

Figure 37 BER performances versus OSNR of CO-OFDM-based FL-VC packetsand single-carrier-

based FL-VC packet

4.2 Cross-talk tolerant multi-carrier payload packet

Another advantage of the MC-based FL-VC packet can be found in the presence of the cross-talk

(XT) between packets. In OPS network, a notable source of XT is the in-band XT in optical

switches OSWs. Extinction ratio (ER) of high-speed OSWs is not always sufficiently high.

Particularly, practical photonic-integrated switch cells for large-matrix switch have ER of 12 to 16

dB [Qian14]. The leakage accumulated from all the cells can result in the non-negligible in-band-

XT. So far, the XT-suppression techniques have been studied in the context of the XT-suppressing

topology [Qian14], where the light paths inside an OSW fabric are carefully chosen to suppress

aggressors. In contrast, in the multi-carrier-formatted OPS network, we can generate optical packets

so as to be resilient to in-band-XTs by cooperatively modulating the subcarriers. A notable



of 78

advantage of such DSP-aided approach is its programmability and adaptability. Particularly it may

be beneficial to mitigate some residual XT, which is difficult to completely remove in the analog-

domain in a cost-efficient manner.

Figure 38 shows the schematic of the proposed XT tolerant packet. The optimal modulation format

for the XT suppression may be formulated as a capacity maximization problem in a MIMO

interference channel. In addition, to cope with the chromatic dispersion and the differential modal

group delay (DGD), it is beneficial to handle the XT in the frequency domain by employing multi-

carrier modulation techniques. Let us consider 𝐼 × 𝐼 switching and suppose the input stream at each

port consists of 𝑀 subcarriers (or sub-bands) whose bandwidth is 𝑊. Here we omit the polarization

channels for simplicity. The averaged power of each subcarrier is controlled between 0 and 𝑃𝑚𝑎𝑥

with 𝑄 discrete levels, i.e., the input power of the 𝑚th subcarrier transmitted from the port I to j

(𝑖, 𝑗 ∈ 𝐼) is represented as 𝑞𝑖𝑗(𝑚) 𝑄⁄ ∙ 𝑃𝑚𝑎𝑥 , where 𝑞 ∈ {0, 1, 2,· · · , 𝑄} . The discrete level is

employed to simplify the implementation of numerical algorithms. For convenience, we note the

path between the input port I and the output port j currently connected as ij and the set of the paths

as Θ. In practice, it is useful to estimate the system capacity based on a signal-to-noise-pulse-

interference ratio (SINR) model. The capacity of the 𝑚th sub-band in the path 𝑖𝑗 can be described as,

𝐶𝑖𝑗(𝑚) = 𝑊 log2 (1 + 𝑆𝐼𝑁𝑅𝑖𝑗(𝑚)) ,

where,

𝑆𝐼𝑁𝑅𝑖𝑗(𝑚) =ℎ𝑖𝑗(𝑚)𝑞𝑖𝑗 (𝑚)𝑃𝑚𝑎𝑥/𝑄

𝜂𝑊 + ∑ 𝑅𝑘𝑗ℎ𝑘𝑗(𝑚)𝑞𝑘𝑗(𝑚)𝑃𝑚𝑎𝑥/𝑄𝑘∈𝐼.

In Eq. 2, 𝜂 is the noise power spectrum density, ℎ𝑖𝑗(𝑚) is the channel gain of the 𝑚th band in the

path 𝑖 → 𝑗，and 𝑅𝑖𝑗 is the XT level between the input port 𝑖 to the output port 𝑗 normalized by

∑ ℎ𝑖𝑗(𝑚)𝑚 . Note that, even though the XT is modeled as frequency-flat for simplicity, the resulting

XT power per subcarrier becomes frequency-selective due to {𝑞𝑖𝑗(𝑚)}. This implies that there is the

optimal set of {𝑞𝑖𝑗(𝑚)} which maximizes {𝐶𝑖𝑗(𝑚)}. For this goal, one may simply maximize the

sum of 𝐶𝑖𝑗(𝑚), i.e., ∑ ∑ 𝐶𝑖𝑗(𝑚)𝑚𝑖 , under an energy constraint. However the maximization may be

useless in practice, because the maximum is often achieved by blocking some paths with poor XT

performance. Therefore we consider to maximize the minimum capacity among the paths to resolve

the capacity bottleneck due to XT;

max{𝑞𝑖(𝑚)}

min𝑖∈𝐼

∑ 𝐶𝑖𝑗(𝑚)𝑚

s. t. ∑ 𝑞𝑖𝑗(𝑚) ∙ 𝑃𝑚𝑎𝑥 𝑄⁄𝑚∈𝑀 ≤ 𝑃𝑡𝑜𝑡𝑎𝑙 , (𝑖𝑗 ∈ Θ)

where 𝑃𝑡𝑜𝑡𝑎𝑙 is the upper-bound of the transmitted optical power per mode. The above equation is a

mixed integer non-linear program (MINLP) and is NP-hard in general. However it is still possible to

solve it numerically by the branch-and-bound method as in [Shi11]. Once the optimal {𝑞𝑖𝑗(𝑚)} are

found, the number of bits per subcarrier is decided based on the SINR to achieve the target BER



of 78

performance. Note that, once the bit-/power-loading map is fixed, no MIMO-DSP at the transceiver

is needed.

Figure 38 b) shows a numerical result where the achievable capacity; 𝐶𝑚𝑖𝑛 = min𝑖𝑗∈Θ

∑ 𝐶𝑖𝑗(𝑚)𝑚∈𝑀 ,

versus SNR in a 2 × 2 optical packet switching is evaluated. Any channel impairments other than

the modal XT are neglected for simplicity. The XT levels are { 𝑅12,𝑅21} = {−14, −23} [dB].

Without the XT suppression, the capacity is restricted particularly in the high SNR region. With the

suppression, the capacity bottleneck is removed and an up-to 22 % improvement is achieved.

Figure 38 a) Image of XT-tolerant multicarrier payload packet and b) the capacity of OSW with 14 and

23 dB XTs with or without the optimization

Figure 39 Experimental setup for a 2×2 switching of 10-Gbaud coherent optical OFDM packets

The feasibility of the proposed XT-tolerant packet is further investigated experimentally. Figure 39

shows the setup for a 2×2 switching of 10-Gbaud coherent optical OFDM packets. As it can be seen

in Figure 40 a), the BER performances of the CO-OFDM packets with the fixed modulation formats,

i.e., QPSK, 16QAM and 32 QAM, can be degraded by the in-band-XT rather than the additive white

noise. In other words, the achievable capacity of the fixed-formatted OFDM packet is limited by the

XT. In such XT-dominant environment, the proposed can improve the achievable capacity by

adapting the loaded bits and power onto each sub-band (Here, the 128 OFDM subcarriers are

divided in to 3 sub-bands for simplicity.) for given XT-levels. In fact, the achievable capacity was

improved 33% in the case with XT = 13 dB and8.25 % for the XT-level of 21 dB. Here the simplest

case with a single switching fabric is considered and demonstrated, however, it is worth noting that

the proposed approach can be extended to multi-hop scenarios in theory as shown in [Shi11].



of 78

Figure 40 Experimental results; (a) BER performances versus ER between the port 2 and 3, (b)

achievable capacity versus the ER between the port 2 and 3, and (c) the resulting bit- and power-loading

map for ER of 23 dB between the port 2 and 3

4.3 Distance-adaptive IM-DD FL-VC OPS based on DMT

The OPS network is an attractive solution for access, aggregation, and/or intra-datacenter networks.

Thus it may also be interesting to define the FL-VC optical packet based on cost-effective intensity-

modulation and direct-detection (IM-DD) systems. In the IM-DD systems over SMF at C-band, a

notable capacity limiting issue is the fiber’s chromatic dispersion. Based on [Barros10], the based

band equivalent of the frequency response of the fiber’s chromatic dispersion after the square-law

detection at the photo-detector (PD) can be approximated by,

𝐻𝐶𝐷(𝜔) = cos {𝛽2

2𝜔2𝐿},

where 𝛽2 is the chromatic dispersion coefficient and L is a transmission distance. Then one may

formulate the ergodic capacity of the channel as

𝐶𝐶𝐷 = ∫ log2 {1 + cos2 {𝛽2

2𝜔2𝐿} ∙

𝑃(𝜔)

𝑁0} 𝑑𝜔

𝑊

0

,

where 𝑃(𝜔) denotes the (one-sided) power spectrum density of the transmitted signal, 𝑁0 is the

noise power spectrum density, and W is the overall bandwidth of the optoelectronic devices, such as

a the laser diode and a D/A converter. From the equations, it is obvious that the channel frequency

response has the more ‘fadings’ for the longer transmission distances, the larger dispersion

coefficients, and/or the wider bandwidth, and the spectrum efficiency is restricted. It is also can be

seen that if one can optimize 𝑃(𝜔) under a certain transmitted energy constraint, the capacity 𝐶𝐶𝐷



of 78

can be maximized. In fact, the maximization problem is the famous ‘water-filling’ interpretation,

and multi-carrier modulation techniques are known as a practical way to realize the water-filling

power spectrum 𝑃opt(𝜔) with feasible transmitter complexities. Therefore, it is reasonable to

employ IM-DD multicarrier-modulation, such as DMT, for the IM-DD-based FL-VC packet to

maximize the capacity of each packet in the presence of the chromatic dispersion. After the bit- and

power-loading in the subcarrier domain, the resulting capacity of the DMT packet depends on the

link distance L, so the packet may be called as a distance-adaptive-type FL-VC packet.

In [Yoshida14], we experimentally evaluate the achievable capacity of the DMT-based IM-DD FL-

VC packet. Figure 41 shows the experimental setup for a 2×3 packet switching of the optical DMT-

based FL-VC packet. The DMT-based packet with 1028 subcarriers is generated via a 65 GS/s DAC

and EML. The resulting bandwidth of all the optoelectronic components is around 30GHz. The

packets have a fixed-length of 283.5 ns and are then switched by a 2×3 PLZT OSW. Figure 42

shows the BER performance versus the SMF link distance after the OSW. In the figure, the

achievable bitrate with the 7% FEC is also depicted, i.e., 108.2 Gb/s for < 2 km and 46.7 Gb/s up to

40 km.

The distance-adaptive DMT-packet can be a spectrum efficient solution in the IM-DD OPS at the C-

band. However, since the payload capacity is depends on the link distance, its contribution to the

entire network performance, such as throughput, is not so obvious. So, we analyze the performance

of the DMT-based FL-VC OPS network theoretically in the next section.

Figure 41 Experimental setup for 2×3 switching of 100Gbps-class optical DMT-based packet



of 78

Figure 42 BER of DMT-based FL-VC packet and achievable bitrate with 7% FEC versus link distance

4.4 Performance analysis of distance-adaptive FL-VC OPS network

We have analyzed the performance of the distance-adaptive DMT-based FL-VC packet in several

OPS scenarios. In the FL-VC OPS, the payload capacities vary from transmission-distance to

distance. Thus the performance of the OPS network based on FL-VC may largely depends on the

geographic distribution of traffic. Moreover, the choice of the packet length may also impact on the

performance. Our methodology has consisted in mathematical analysis and computer simulation. As

both techniques allow simplifying the study of the interactions of the many entities interacting

through the network, we have chosen to deal directly with user-centric performance metrics. As

most applications run over TCP, flow completion times (FCTs), or equivalently application

throughputs, are typically the only metrics that matters to them. Thus, we have focused our study on

these metrics.

4.4.1 On packet capacity distributions

The choice for the length (or size) of optical packets is critical to entire network performance in

FL-VC OPS. Effective packet durations can be selected from the analysis of empirical traffic

patterns. Packet capacities in many packet switching networks are quasi-bimodal in terms of their

frequency of appearance. As an example, Figure 43 depicts the byte and packet cumulative

distribution functions of the traffic seen at several busy Internet Service Provider (ISP) exchange

points in 1998, 2008 and 2014[CAIDA08, WIDE15]. By 2008, the 576-byte mode has practically

disappeared, and thus packet capacities only appear in significant numbers in one of two values: the

50-byte one corresponding to TCP acknowledgment packets (ACK), and, and the 1500-byte

Ethernet maximum transmission unit (MTU). These two packet capacities appear roughly with the

same likelihood, and together comprise more than 80% of all traffic. This bimodal behavior is not

caused by any characteristic specific to the particular technology used in the depicted ISPs. Rather,

its ultimate cause is the interplay between two factors residing at the ends of the connections,

outside the OPS network: the sizes of the files stored at the end hosts and the behavior of TCP, the

transport protocol commonly used to guarantee ordered, reliable service across best-effort packet-

switching networks. To illustrate the first factor, we plot in Figure 44 the probability density



of 78

function of the file sizes found in three scenarios: two data centers specialized in data-mining

[Greenberg09] and web-search [Alizadeh10] applications, respectively, and a busy file server in a

local area network (LAN) [Agrawal07]; scenarios like these also determine the characteristics of an

important part of the flows exchanged across MAN or wide area networks (WAN). Although

different, the three distributions shown in Figure 44 share a heavy-tailed nature shown in the

significant presence of very large files: although their likelihood of appearance is small compared to

small files, it is large enough to contribute most bytes transported across the network, as seen in the

byte-related lines in Figure 43. As most packets belong to relatively few large files, their capacities

will be maximal (i.e., the MTU) except for the last one of each transmission. In addition, due to TCP,

there will be one small ACK packet for each data packet or for each two data packets. As a result,

most packets end up having either very large or very small capacities, with few packets taking

intermediate values.

Figure. 43 Trimodal packet capacity distributions in 1998 (Ethernet MTU, IP minimum MTU,

and ACKs), and bimodal in 2008 and 2014 (Ethernet MTU, ACK) [CAIDA08, WIDE15]

Figure 44 File size distributions in data centers and LANs [Greenberg09, Alizadeh10,

Agrawal07]

4.4.2 Achievable throughput by fixed-length optical packets

For simplicity, first, we analyze the theoretical throughput of the FL-VC OPS apart from network

topology and geographical traffic distribution. To simplify OPS packet generation, there are two



of 78

main ways of leveraging this bimodal characteristic. The first is to use a constant duration for all

packets, or hereafter monomodal packets. The second is to use two different packet durations, one

for data packets and another for ACKs, or hereafter bimodal packets. We depict in Figure 45 a

diagram of these two modes.

Figure 45 Diagram of the two options: monomodal packets (top) and bimodal packets

(bottom). 𝛄 denotes the data packet duration, and 𝛅 the round-trip propagation time.

Figure 46 Diagram of stop-and-wait congestion control [Losada14].

Although the bimodal case can offer advantages related to a better adequacy to empirical packet

patterns, it is significantly more complex to implement than the monomodal case. As we will see

next, both modes lead to significant increases in application throughput with respect to conventional

technology. Thus, here we advocate using a single constant packet duration for FL-VC packets to

optimize the trade-off between network performance and implementation ease.

Let us consider a file size distribution 𝑆 where a file of size 𝑖 appears with probability Pr (𝑖|𝑆). The

constant FL-VC data packet duration is denoted by γ, and the ratio between ACK packet duration

and data packet duration is denoted by 𝜌 (if all packets have the same duration, then 𝜌 = 1). Let us

first assume that there are no losses in the OPS network, and that the round-trip propagation time δ

is constant. The receiver sends an ACK packet to the sender for each data packet it receives, and

the sender sends one packet at a time (i.e. stop-and-wait as implemented by TCP versions designed

for OPS networks [Losada14]) as in Figure 46. If 𝑐 is the bit rate for the packet payloads, then the

flow completion time of a given flow transmitting a file of size 𝑠 bytes will be ⌈8𝑠

𝛾𝑐⌉ ∙ (γ + γσ + δ),

where ⌈8𝑠

𝛾𝑐⌉ is the number of data packets comprising the file. Therefore, the average throughput of

an application transmitting a file chosen from S using fixed-length OPS packets across a zero-loss,

constant-delay path is



of 78

ΔFL = ∑ 𝑃𝑟 (𝑖|𝑆) ∙

𝑖∈𝑆

8𝑠𝑖

⌈8𝑠𝛾𝑐⌉ ∙ (𝛾 + 𝛾𝜎 + 𝛿)

.

In contrast, conventional network technology using packets with variable lengths (VL) whose MTU

is 𝑀 bytes will divide the file into ⌈𝑠

𝑀⌉ packets. After each packet is sent, an acknowledgment arrives

after 𝛿 units of time. Each acknowledgment lasts 𝜌 ∈ (0, 1) times the duration of an MTU packet.

The flow completion time of a file of size 𝑠 bytes can then be approximated as 8𝑠

𝑐𝑙+ ⌈

𝑠

𝑀⌉ ∙ (𝛿 +

8𝑀

𝑐𝑙𝜌).

Therefore, the average throughput of conventional OPS technology, 𝛬𝑉𝐿, can be computed as

𝛬𝑉𝐿 = ∑ 𝑃𝑟 (𝑖|𝑆)

𝑖∈𝑆

∙8𝑠𝑖

8𝑠𝑐𝑙

+ ⌈𝑠𝑀⌉ ∙ (𝛿 +

8𝑀𝑐𝑙

𝜌)

We plot 𝛬𝐹𝐿 and 𝛬𝑉𝐿$ in Figure 47 as a function of the packet duration 𝛾 when the round-trip

propagation delay corresponds to 1km (i.e., 𝛿 = 5𝜇𝑠) and for the common packet bit rate of 43Gb/s.

The MTU for the variable length case is 𝑀 = 1500 bytes, as customary in Ethernet deployments.

𝛬𝑉𝐿 is represented by the horizontal line at the bottom of the figure. 𝛬𝐹𝐿 is represented by the rest of

curves: the monomodal case in white-point lines, and the bimodal case with ACKs 100 times shorter

than data packets, i.e., 𝜌 = 0.01, in black-point lines; there is a line for each of the three file size

distributions shown in Figure 43.

Figure 47 Throughput achieved in the transmission of a file across a 1km-long path with

packet bit rates of 43Gb/s.

The benefits of using fixed-length packets are readily apparent: packet durations in the interval 𝛾 ∈

(10, 100) 𝜇𝑠 greatly increase the throughput of the file transmission due to its adequacy to the file

size distribution function. In this way, the throughput of the connection improves between 4 and 18

times with respect to conventional OPS technology using variable duration packets with the limited

MTU of 1500 byte. Note that, obviously, the variable duration packets with much larger MTU size

may achieve the better performance. However it requires additional complexity to OPS nodes to

handle large variety of packet durations.



of 78

4.4.3 Achievable throughput by the distance-adaptive FL-VC packets

Each OPS packet traverses a path whose length can be in a wide range of values, from very short

(e.g., several meters) to much longer (e.g., tens of kilometers). Additive noises and dispersion cause

signal impairments of packets, and these impairments worsen with distance. For single fiber spans,

they usually translate into an approximately constant bandwidth-distance product that leads to an

inversely proportional relationship between fiber length and the maximum bitrate achievable. Thus,

in order to enable full connectivity in an OPS network where packet regeneration is not available, it

is advantageous to adapt the modulation format of each packet to the characteristics of its expected

path.

As an example, let us consider an OPS network where the maximum path length is 40km, and the

minimum is 1km. The achievable capacity for a given distance is assumed to be the same as our

experimental results in [Yoshida14] (or Figure 42); 101Gb/s in distances under 2km; rates must then

decrease to 43Gb/s when distances grow to 40km. For comparison, throughput of an OPS network

using conventional variable-length, fixed-bit-rate (VL-FBR) packets is also considered. The bitrate

of VL-FBR packets need to be fixed to 43Gb/s to guarantee global connectivity. Figure 48 shows

the theoretical throughput for transmitting one file achieved by either FL-VC or VL-FBR OPS

where the path is 1km long. The throughput gain with respect to conventional transmission can go

up to ~35 times for the bimodal packets case or to ~20 times for the monomodal packets case, due to

a combination of higher bit rates (VC) and suitable packet duration (FL).

Figure 48 Throughput transmitting one file achieved by either FL-VC at 101Gb/s or

conventional, variable-length (VL) OPS transmission at 43Gb/s. The path is 1km long.

4.4.4 Performance analysis in MAN

Here we extend our previous theoretical analysis to a MAN shared by multiple flows, where each

path can have a different length and where thus the packet capacity can be different for each flow.

We will focus on the average throughput of multiple connections using FL-VC packets across the

network. In what follows, we assume that the network does not use WDM for simplicity, although

FL-VC packets can be readily used in WDM networks as well.



of 78

If a network has 𝑛 hosts, each of the 𝑛 − 1 end-to-end paths starting at a host can in general operate

at a different bit rate. Let us thus define 𝜔(𝑖, 𝑗) as the maximum achievable bit rate in route 𝑖 → 𝑗,

𝛺𝑖, 𝜔𝑖 as the maximum and minimum respectively of the set of bit rates for routes starting at node 𝑖,

and 𝛺, 𝜔 as the maximum and minimum respectively of the set of all bit rates in the network.

A conventional OPS transponder generating VL-FBR packets at node 𝑖 uses the same bit rate for

all packets; that rate must thus be 𝜔𝑖 in order to guarantee connectivity to all routes. In contrast, a

FL-VC transponder uses either the VL-FBR rate or one of the higher bit rates. The maximum

achievable bit rate increase allowed by FL-VC from node 𝑖 is thus Ω𝑖/𝜔𝑖, and it is achieved in the

routes with the highest-quality optical paths; the remaining routes obtain increases in the range

[1, Ω𝑖/𝜔𝑖 ]$. The average achievable bit rate increase among all network flows depends on the

distribution of traffic across all routes. Let us thus define the traffic matrix describing the offered

traffic to all routes in the network as Θ ≜ |𝜃(𝑖, 𝑗)|, where 𝜃(𝑖, 𝑗) is the relative amount of time used

by the transponder at node 𝑖 to send traffic to node 𝑗, and where we assume 𝜃(𝑖, 𝑖) = 0 ∀𝑖. For

simplicity, let us assume that 𝛺𝑖 and 𝜔𝑖 are the same for all sending nodes, i.e., 𝛺𝑖 ≡ 𝛺, 𝜔𝑖 ≡ 𝜔,

and therefore all transponders are identical in either the VL-FBR or FL-VC cases. VL-FBR

transponders have a bit rate equal to the minimum of the bit rates supported by FL-VC: 𝜔 =

min𝑖,𝑗

𝜔(𝑖, 𝑗). We define 𝛥𝑖 as the average bit rate improvement obtained by replacing the VL-FBR

transponders with FL-VC ones in routes starting at host 𝑖, and 𝛤 as the average gain in all the

network. Then,

𝛥𝑖 = ∑𝜃(𝑖, 𝑗)𝜔(𝑖, 𝑗)

𝜔

𝑁

𝑗=1

, 𝛤 ≜ ∑𝜃(𝑖)

∑ 𝜃(𝑗)𝑁𝑗=1

𝛥𝑖,

𝑁

𝑖=1

where we define 𝜃(𝑖) as the sum of the elements in row 𝑖 of 𝛩: Θ ≜ ∑ 𝜃(𝑖, 𝑗)𝑁𝑗=1 . Note that 𝛤 is a

scalar in the interval 1 ≤ 𝛤 ≤ 𝛺/𝜔.

Figure 49 MAN ring topologies

Let us quantify the above considerations by means of several examples intended to show how the

performance of FL-VC scales with network size in common MAN scenarios. We consider three

typical bidirectional ring topologies, depicted in Figure 49: a small ring of 4 switches with 2km links,



of 78

a medium 12-switch ring with 3-node clusters and 10km inter-cluster links, and a large 28-switch

ring with 2-node clusters and 2km inter-cluster links; all in-cluster links are 1km long.

We consider the following three traffic matrices:

1) Uniform traffic (UT):This case represents a network where all routes appear with the same

probability.

2) High-locality traffic (HL): Network design often tries to locate traffic sources as close

aspossible to their destinations: e.g., cache servers for popular content in video portals [GoogleGC,

AkamaiC]. Institutions also tend to concentrate services that interact with each other in close

geographical areas. These effects make short routes prevalent over long ones. Here we thus consider

a traffic matrix where 90% of the traffic stays inside clusters, while the remaining 10\% is uniformly

sent to the rest of clusters (clusters in the small ring are composed of the 2 closest nodes to a given

one); all nodes inside a cluster have the same probability of getting or receiving traffic.

3) Hot-spot traffic (HS): Sometimes a node sends or receives a significant amount of the traffic in

the network; e.g., when acting as a gateway to popular routes. These hot-spot situations tend to be

avoided by network planners due to their criticality, but can nevertheless appear during normal

operating conditions. We thus consider here a host receiving 90% of all the traffic sent by all nodes;

the remaining 10% is uniformly distributed among the rest of routes.

Figure 50 Theoretical bit rate gain.

Figure 50 accordingly depicts 𝛤 for the topologies and traffic matrices described above. We use the

bit rate of an OPS network using conventional, VL-FBR transponders as the reference, with a value

of 1; this corresponds to 43Gb/s, the minimum bit rate in Figure 42. Let us focus first on the medium

and large rings. FL-VC obtains higher bit rates in them than VL-FBR particularly for the HL traffic

matrix: around 2.3 times. The HS and UT cases do not favour short routes as much as the HL one,

and thus obtain smaller gains; these are nevertheless very significant: between 40% and 50% for the

medium and large rings. The HS and UT cases obtain essentially the same performance because,

although their traffic patterns are very different, they have similar proportions of traffic in short,

medium and long routes. Regarding the small ring, where the FL-VC transponder can almost always

employ the highest bit rate of 101Gb/s, we can see that its HS gain is considerably larger than in the

other topologies. Although this can be seen at first as an unfair comparison, it can still happen since

networks are often built taking into account potential expansions.



of 78

We emphasize that these high bit rates cannot be achieved by conventional, VL-FBR transponders:

On the one hand, if all of them use the lowest bit rate available to all routes (43Gb/s), throughput is

too low. On the other hand, if they all use the highest one (101Gb/s), only the nodes that are closer

than 2km can intercommunicate, with longer routes unable to provide connectivity due to

impairments, and end users finding most routes unresponsive. In contrast, FL-VC obtains large

throughput increases without breaking network connectivity, due to its bandwidth-variable

transponder technology.

As a validation of the previous analysis, we plot in Figure 51 the 95% confidence intervals (CI) for

the throughput obtained by simulation of UDP traffic in the 12-node ring. We consider the high-

locality, hot-spot and uniform traffic matrices. As the theoretical analysis had suggested, the

advantages of FL-VC over conventional transmission are distinctly clear, allowing a noticeable

increase in the traffic carrying capabilities of the network. The high-locality traffic matrix also leads

to the highest throughput gains, as expected. The other two traffic matrices do not use short routes as

often as the first one, so they do not benefit from the higher bit rates so much. As empirical traffic

patterns and network design procedures often tend to produce high-locality characteristics, it is

expected that FL-VC will lead to great throughput improvements in practice.

Figure 51 FL-VC vs. VL-FBR: simulation of UDP traffic in the 12-node ring.

4.4.5 On the designing of distance-adaptive FL-VC OPS network

Here we further investigate the performance of the distance-adaptive FL-VC OPS networks and

discuss the optimal choices for packet duration, congestion control method, and the size of optical

buffers, each of them is critical to the entire network performance.

Network performance can be measured according to several metrics, traditionally categorized in two

groups: user-centric and network-centric. As the user-centric ones are the ultimate ways of assessing

the effectiveness of any network, we will focus on them, choosing a measure of how fast is the

network when transferring a group of files on a given set of routes. Let us consider a group of 𝑛 file

transmissions with sizes 𝑠𝑖 ∈ {1, 𝑛} arriving at rate 𝜆 at the network edge and whose sources and

destinations are chosen according to an arbitrary traffic matrix. We accordingly define the network

throughput Ψ of this set of file transmissions as the sum of their file sizes divided by the time



of 78

elapsed between the start of the first transmission (we assume it is time zero) and the end of the last

one, 𝑇:

Ψ ≜∑ 𝑠𝑖

𝑛𝑖=1

𝑇.

Measuring performance using 𝛹 makes most sense to applications, since they are mostly interested

in their flow completion times. It is also useful to the network operator, since it offers a direct

measure of how fast it can attend user requests for bandwidth, and thus a much more direct indicator

of the quality-of-service received by users than other metrics like losses or utilizations.

We purposefully avoid the usage of network-centric performance metrics. These are often used in

the literature because they are relatively straightforward to measure; however, they are not

meaningful per se to applications once flow completion times are specified. These metrics include

losses, sent or received bit rates, individual packet end-to-end delays or jitter, or link utilizations.

Losses, for example, are immaterial to applications since TCP is in charge of offering a lossless

service: losses are only apparent to most applications by means of how the throughput offered by

TCP reacts to them, which usually involves a reduction of the sending rate.

As we are measuring the flow completion times, we are already taking this issue into account.

Delay-based metrics applied to individual packets like end-to-end delay or jitter are also often

unsuitable to predict application performance since applications typically need the whole transfer to

finish before proceeding to their next steps; e.g., it does not matter that many packets experience

very low end-to-end delay or reduced jitter if one of them gets significantly delayed and the entire

flow cannot finish on time for a deadline. Sent or received amount of bits in a given period of time

are problematic as well since part of these can correspond to retransmitted packets which do not

count towards meaningful application throughput. In essence, network-centric metrics only matter to

most applications in the extent at which they affect their completion times. Assessing a network

scenario by means of network-centric performance metrics is thus often misleading: e.g., a network

with low losses and high utilization can be giving either a very good service to applications or a very

poor one, and it is not possible to know which one it is until taking a look at a relevant user-centric

performance metric, and in most cases this metric is the flow completion time. We emphasize that

this aspect is particularly important for assessing OPS networks. In OPS networks, most packets are

processed on-the-fly due to the limited optical buffering capability. Hence they experience almost

negligible delay in the physical layer while their loss probability is higher than that of in electrical

packet switching networks.1) Optimal FL-VC packet duration: We depict in Figure 52 the 95%

Cis for the network throughput Ψ as a function of the packet duration 𝛤 when transferring 1000

random files across the 12-node MAN ring with the high-locality traffic matrix. White-point and

black-point lines plot the throughput ΨFLVC achieved by bimodal and monomodal FL-VC packets

respectively. Flat lines depict the throughput ΨVLFBR of conventional VL-FBR packets at 43Gb/s

and with a MTU of 1500 bytes. We plot the curves corresponding to the three file size distributions

in Figure 44. The FL-VC network is asynchronous (not slotted). As the OPS switches we consider

do not have buffers. We use TCP SAW in Figure 46, a TCP version that overcomes this high packet

losses by leveraging the low-delay that is also characteristic of bufferless networks and that allows

data sources to react to losses faster than in networks with buffers. TCP SAW uses stop-and-wait as



of 78

its congestion control algorithm, sending one packet at a time and waiting for its acknowledgment.

A retransmission timer using the current estimate of the round-trip-time and exponential back-off is

in charge of assessing losses. The key difference between all-optical and conventional networks, i.e.,

the lack or presence of random-access-memory buffers in their switches, allows to have round-trip-

times that are much smaller in the former than in the latter, and consequently much better

responsiveness and throughput.

Figure 52 FL-VC vs. VL-FBR: Simulation of TCP traffic in the 12-node ring with the high-

locality traffic matrix.

In Figure 52, we can see how an adequate selection of the FL-VC packet duration can greatly

maximize network throughput with respect to VL-FBR as in the previous sections. The packet

durations that maximize throughput are in the range ≃ [3, 30]μs for the monomodal case and in ≃

[30,100]μs for the bimodal one. The throughput increase that FL-VC achieves over conventional

VL-FBR in these ranges, α ≜ ΨFLVC/ΨVLFBR, depends on the workload distribution and on whether

bimodal or monomodal packets are used. For monomodal packets, the maximum α are 3, 8 and 16

for the file-server, web-searchand data-mining distributions respectively. For bimodal packets, the

corresponding maximum α are 10, 16 or 32 for the file-server, web-search and data-mining

distributions respectively as well.

2) Effect of the TCP congestion control, file arrival rate and file size distribution: Figure 53

depicts the influence of several variables on the FL-VC network throughput Ψ: the TCP congestion

control algorithm, the arrival rate of file transmission requests 𝜆, and the file size distribution. FL-

VC packet duration is γ = 35 μs, the value that maximizes throughput in Figure 52 for the file-

server distribution in the bimodal case. White-point lines correspond to TCP SAW, and black-point

lines to TCP SACK [Mathis96], a TCP version often found in most operating systems. Their

comparison readily shows that selecting an adequate TCP version is very important to overcome the

penalties associated to the bufferless environment without the need to resort to access control or

complex scheduling algorithms. TCP SAW is able to consistently outperform TCP SACK by several

orders of magnitude for medium and high file transmission request arrival rates 𝜆 . We emphasize

that in order to have a meaningful comparison of these TCP versions it is necessary to measure the

completion times of the flows as opposed to other metrics such as losses, delays, utilizations, etc.



of 78

Figure 53 Influence of the TCP congestion control algorithm, connection arrival rate and file

size distribution on the throughput of the 12-node FL-VC MAN ring with the high-locality

traffic matrix.

Figure 53 also allows to readily grasp how the file size distribution affects performance. The circular

points represent the case of constant file sizes and the square points the case of lognormal file sizes;

these last ones correspond to the file-server distribution in Figure 44. As we had previously noted

when discussing Figure 52 and the three empirical file size distributions used there, these two

additional distributions also obtain different throughputs. However, in Figure 53 as in Figure 52, the

difference among them is not particularly high, confirming that the selection of FL-VC packet

duration is robust and stable against changes in the workload. This is a significant advantage given

that workloads can be difficult to predict and can also change rapidly during normal network

operating conditions.

3) Impact of buffer size: Our previous experiments used an OPS network where switches lacked

buffers of any kind. In contrast, here we will focus on the performance of the network when fiber-

delay-line (FDL) buffers are available in the switches. FDLs offer fixed delays to packets, and can

induce reordering effects inside TCP flows. This is in general undesirable because some TCP

versions can mistake reordering events for loss events; in our case, however, TCP-SAW only has

one packet in flight at a time and consequently cannot suffer reorder. Moreover, even if other

versions of TCP more sensitive to reorder than SAW were to be used, OPS networks have such

small delays as a consequence of their lack of RAM buffers that the likelihood of significant

amounts of reordering inside a flow is extremely small compared to their electronic counterparts.

Figure 54 and Figure 55 accordingly depicts the network throughput in several illustrative cases

when the file arrival rate is λ = 106𝑠−1 in the case of the file-server workload. FDL lengths are

consecutive multiples of the FL-VC packet duration of 35μs. We consider two schedulers that

represent two extremes in terms of their feasibility. On the one hand, a first-fit scheduler that selects

the shortest available FDL when a packet finds an output fiber busy; we consider either 1 or 8 FDLs,

since they have already been successfully implemented in prototypes; recirculation is not allowed

due to potential physical impairments. For comparison, we also consider a much more complex

scheduler where unlimited recirculation of packets is allowed and that can manage up to 256 FDLs.

Figure 54 shows the results for the first-fit scheduler, and Figure 55 shows the results for the

unlimited recirculation one. 1000 files are transmitted across the network in each experiment.



of 78

Figure 54 Effect of FDLs on the throughput: FDL without recirculation, first-fit scheduler, and

𝛌 = 𝟏𝟎𝟔𝒔−𝟏

Figure 55 Effect of FDLs on the throughput: FDL with recirculation, and 𝛌 = 𝟏𝟎𝟔𝒔−𝟏

We can observe that FDLs allow in general to improve performance. In the case of the first-fit

scheduler, the throughput improvement is up to ≃ 60%, achieved with a minimum FDL length of

≃1 packet duration. In the case of the complex scheduler, allowing recirculation makes the optimal

FDL length one packet long or smaller; any value in that range achieves similar performance. There

are no appreciable differences between having 8 and 256 FDLs either, suggesting that nearly-

optimal performance can already be achieved with very few FDLs. In summary, we can conclude

that the throughput improvement caused by even a single FDL managed with a simple scheduler is

very noticeable.

4.4.6 Latency considerations in FL-VC OPS network

In this section we perform a comparative study of the influence on network latency of FL-VC and

conventional FBR-VL. Latency can measured in multiple ways in any network. As the network can

frequently become a bottleneck for applications, here we will use the point of view of a distributed

application using the network to exchange a set of files needed for its internal logic. Thus, we define

the latency of a group of files that need to be transmitted across the network according to a given



of 78

traffic matrix as the time elapsed between the beginning of the first file transmission until the

completion of the last one. This way of defining latency is especially useful because it allows a

direct comparison of different network technologies (i.e., FL-VC vs. VL-FBR) from the point of

view of the entities for whom the network is built in the first place (i.e., the applications).

Figure 56 FL-VC vs. a network with RAM buffers in the 12-node MAN ring scenario with the

high-locality traffic matrix and the file-server workload.

We accordingly plot in Figure 56 the latency of the transmission of 1000 files in the 12-node MAN

ring shown in Figure 49 and according to a traffic matrix where 90% of all flows stay inside a

cluster and the remaining 10% are randomly sent to nodes in other clusters. Simulation has been

used to obtain the measurements, and confidence intervals are so small that they do not appear in the

figure. We consider the bimodal FL-VC packet durations. We plot the latency as a function of the

FL-VC packet duration for the file-server, web-search and data-mining distributions described in

earlier sections. We also plot in horizontal lines the corresponding latencies for conventional VL-

FBR at 43Gb/s with MTU of 1500 bytes.

We can clearly see in Figure 56 how an adequate choice for the FL-VC packet duration allows a

noticeable reduction in latency for applications. For example, with the file-server distribution,

conventional VL-FBR imposes a latency of ≃100ms on the distributed application before allowing it

to proceed, and FL-VC can decrease that time one order of magnitude, i.e., down to 10ms. For the

web-search application, VL-FBR latency is ≃2s, and FL-VC is able to decrease it more than one

order of magnitude, down to ≃100ms. Finally, the data-mining application experiences a latency of

a little over ≃10s with FBR-VL, and FL-VC reduces it nearly 2 orders of magnitude, down to

≃200ms. In summary, FL-VC is effective at achieving noticeable improvements regarding network

latency. This latency is often an important bottleneck for applications, which are waiting for those

transfers to finish before being able to proceed (e.g., to return information to users elsewhere as in

common cloud computing applications). Thus, we can see that FL-VC offers a significant

improvement on conventional FBR-VL technology for next-generation optical networks.

Note that, another source of the latency can be found in the physical layer, i.e., processing delay for

DMT modulation and so on. Table 7 shows a list of the estimated overheads for DMT functions.



of 78

Even though the DMT scheme requires additional processing for modulation and de-modulation,

e.g., FFT, compared with the conventional OOK formant, the most of the latencies, excepting that

for the probing, are much smaller than the application delay and thus may be negligible in practical

situations. As for the probing, it is to acquire the channel state information (CSI) for the adaptive

modulation and may take time. However, the physical links in an OPS network is stable and hence

the CSI can be acquired a priori.

Table 7 List of latencies for each DMT function

Item Latency Unit Memo

Probing < 50 ms msFor the system start-up sequence

only

DMT mod./demod. ~ 200 ns

FEC

Low latency

mode~ 200 ns

Depends on FEC selection

High gain

mode~ 20 μs



of 78

4.4.7 Energy consumption of intra-datacenter OPS networks

Figure 57 Intra-DC network employing HOPRS at nodes

We will investigate the energy consumption of intra-DC OPS networks from the viewpoints of both

the hardware and the network topology. It’d be interesting to compare the energy consumption and

the throughput with those of electronic packet switching (EPS). One of typical network topologies

of current intra-DC networks is a fat-tree topology but we don’t know if the fat-tree topology is

suitable for the case with OPS. Here, we focus on N-dimensional torus topology, which is used in

super-computers such as an N-dimensional torus topology, similar to current supercomputers such as

CRAY (3-D), Blue Gene (3∼5-D), and K Computer (6-D) [Yokokawa11]. Note that, in STRAUSS,

we mainly consider the novel optical payload format for OPS networks, i.e., FL-VC, and develop its

generation, reception and controlling techniques. Any OPS node architectures can be employed as

long as they enable transparent optical packet switching.

Figure 57 shows the architecture for our proposed DC network with a HOPR at each node that

adopts an N-dimensional torus topology. For the illustration purpose, 2-D torus topology is shown in

Figure 58. The architecture features a flat architecture where each HOPR is connected to the

neighboring HOPRs through 100 Gbps optical links. Each HOPR is also connected to a group of

top-of-rack (ToR) switches through 10 GbE links. A packet from a server via ToR switch is injected

to the HOPR after undergoing a conversion to a 100 Gbps burst-mode (BM) optical packet (25

Gbps×4λ) by being attached with an optical fixed-length baseband label.

Compared to the conventional fat-tree DC network, the torus network can provide several

advantages, including the superior scalability demanded for the alternate of a huge-capacity core

router and the capability of maintaining robust connectivity even in the case of node failure by

providing with several alternative routes to the destination. On other hand, it is a draw- back that the

torus network needs multi-hop transmission, and thus, it necessitates a dramatic reduction in



of 78

HOPR’s input-to- output delay to reduce end-to-end latency. The discussions on the power

consumption of torus topology versus conventional fat-tree topology will be detailed later on.

Figure 58 Architecture of hybrid optoelectronic packet router (HOPR)

For example of the OPS switch fabric, an architecture of hybrid optoelectronic packet router

(HOPR) is shown in Figure 58. It consists of an optoelectronic label processor, an optical switch,

fiber delay lines (FDLs), and an electrical shared buffer that utilizes a CMOS random access

memory (RAM). Every incoming packet first passes through the label processor, where the

processor figures out the packet destination and checks for contention, and then determines the

corresponding switch output port. In case of no contention, the packet is directly routed through the

optical switch to the desired output port without buffering, thus a very low latency is achieved.

Otherwise, the contention is resolved by performing a deflection routing for the packet. An FDL and

the electrical shared buffer are also available as other alternatives for contention resolution that

provide more flexibility and reliability but at the expense of increasing the end-to-end latency.

The first prototype of HOPR was released in 2009 [Takahashi12], which provided an 8×8 switching

capability for 10 Gbps optical packets with the total power consumption and latency of 360 W and

380 ns, respectively, resulting in 2.25 W/Gbps or 2.25 nW/bit. The new prototype is upgraded 100

Gbps, and it is characterized by drastic improvement in the power consumption and the throughput,

0.09 W/Gbps (120 W/1280 Gbps) and 100 ns, respectively. In order to realize such a high target,

NTT Laboratories have been improving as well as newly developing the set of enabling devices and

subsystems such as the label processor, optical switch, and CMOS buffer etc. as is summarized in

Figure 59.



of 78

Figure 59 Key enabling devices and subsystems of new prototype of HOPR

Let us compare two types of three-tire network in terms of the power consumption and the

throughput; one is the fat-tree topology and the other is 6-D (N=6) torus topology in Figure 60. As

we consider two cases of EPS and OPS, there are four types of packet switching networks. The

assumptions are the followings; there are T ToR switches, and each ToR switch accommodates 20

servers. The aggregation switch connects with 10 ToR switches with 10 GbE working links, and it

also connects with another 10 ToR switches with 10 GbE protection links. There are two core

routers, including the one for the protection. They are connected with the aggregation switches with

100 Gbps links, and each core router has a single 400 Gbps link. The differences between two

topologies are found in the connections of the core routers and the aggregation switches. In the fat-

tree topology, all the aggregation switch connects with the two core routers with working and

protection links, and there is no connection with each other. In the torus topology, on the other hand,

the core router is not connected with all the aggregation switches but is connected with B

aggregation switches out of total T/10 aggregation switches. The aggregation switch connects with

other 2N aggregation switches via 100 Gbps links. The throughputs of the core router and the

aggregation switch in the fat-tree topology, whichever they are optical or electrical, are given by

Core : (100Gbps´T /10 + 400Gbps)´ 2

Aggregation: (10Gbps´10 ´ 2 +100Gbps´ 2)´ 2 = 800Gbps

while the throughputs of the core router and the aggregation switch in the N-D torus topology are

given by

Core : (100Gbps´B+ 400Gbps)´ 2



of 78

Aggregation: (10Gbps´10 ´ 2 +100Gbps´ 2 ´N)´ 2

Figure 60 Fat-tree topology and N-D torus topology

To evaluate the power consumption, the following assumptions are introduced as follows. The

power consumption of the routers and switches are assumed as described in the second column of

Table I. The values of EPS are based upon those of typical current high-end electrical router and 1

Tbps-class electrical switch. In OPS 0.09 W/Gbps is set equal to the target value of HOPR, but 0.3

W/Gbps of the optical core router is hypothetical. The number of ToR switches are T = 20 k, and the

numbers of connections between the aggregation switches and the core router in the torus topology

is B = 200. Note that the torus dimension can be assumed to be N = 6 without a loss of generality

because there will not much difference in the power consumption and the throughput with respect to

the dimension. Here, the power consumptions of the protection links and the core router for the

backup are not taken into account. As a consequence, the total number of servers amounts to 400k(=

20T). We also assume that the power consumptions of SFP, 10 GbE transceiver and CFP, 100 Gbps

transceivers are 1.5 and 20 W, respectively. These values are determined by taking into account the

current typical values and the further improvement to be expected in a few years.

Table I Evaluated power consumption Table II Evaluated throughput



of 78

The evaluated total throughputs of the core routers and the aggregation switches for the fat-tree and

6-D torus topologies in the cases with EPS and OPS, are summarized in Table II. For the simplicity

in the comparison, the throughput of core routers becomes much larger than those in real-world DCs

but the power consumption can be evaluated without any loss of generality. From Table IV, the fat-

tree topology requires much larger throughput for the core router than the case with the 6-D torus

topology, while there is not much throughput difference in aggregation switches between two

topologies. The throughput of the core router in the fat-tree topology is over 400 Tbps, and it would

be unrealistic for the OPS to realize such a huge throughput with decent power consumption. This

is because a single optical router with 2000 (= T/10) ports will not be feasible, and hence, a multi-

stage architecture such as Clos switch will be needed, but it will require optical amplifiers to

compensate the losses. Therefore, the OPS in the fat-tree topology will be out-of-scope for the

present comparison, and hereafter, we will compare the remaining three types. To benefit the

scalability, redundancy, and routing ease of the torus topology and to mitigate the adverse effect of a

large latency due to the multi-hop transmission, the reduction of power consumption as well as the

latency of OPS switch are our primary concern. Although, we are quite sure that targeted power

consumption of 0.09 [W/Gbps] and the latency of 100 ns of the aggregation switch can be achieved

with the current HOPR prototype, there remains uncertainty of the core router’s 0.3 [W/Gbps] and

its latency. The above discussions are summarized in Figure 61.

Figure 61 OPS vs. EPS in Fat-tree topology and 6-D torus topology

5 FGPA-based real-time OPS/OCS integrated interface

The interfacing between OPS and OCS systems is carried out using ultra-high performance

optoelectronics platforms of FPGAs. The ultra-high data rate OPS and OCS communications can be

set up over fixed- or flexi-grid channels, so the FPGA-based development boards are equipped with



of 78

variable-bandwidth transceivers, and the FPGA-based interface design and implementation provides

functional blocks for Ethernet based packet to/from circuit conversion, by exploiting on chip and off

chip electronic components.

Taking into account the system requirements and constraints, we used FPGA-based development

boards from Hitech Global, HTG-V6HXT-X16PCIE and HTG-V6HXT-100G, for the OPS/OCS

integrated interface implementation. Both of the boards powered by Xilinx Virtex-6 HX380T FPGA

device and integrate the most fundamental electrical and optical interfaces for building 40G/100G

subsystems for networking and high-speed applications. We also used several daughter cards for

FMC-to-SFP+ and Airmax-to-SFP+ conversion.

a) b)

Figure 62 a) HTG-V6HXT-100G board; b) Airmax-to-SFP+ extender card

The HTG-V6HXT-100G board with Airmax-to-SFP+ extender card are shown in Figure 62 that

features with 12 transmit and receive lanes @11.3Gbps, 1GB flash, 4x18MB QDRII and 2x72Mb

DDRIII memories.

The HTG-V6HXT-X16PCIE board with FMC-to-SFP+ extender card are shown in Figure 63. The

platform supports two QSFP+ and 6 SFP+ optical connectors, up to 16GB of DDR3 SO-DIMM, and

x16lanes PCIe Gen2 interface. The on-board FPGA Mezzanine Connectors (FMC) along with FMC

modules expand the functionality of the board for different applications.



of 78

a) b)

Figure 63 a) HTG-V6HXT-X16PCIE board; b) FMC-to-SFP+ extender card

Based on the requirement of the system and the FPGA-based platforms constraints, the FPGA-based

OPS/OCS interface design architecture is shown in Figure 64 which includes multiple functional

blocks implemented in the FPGA. The work flow follows the direction of arrows. The blocks in

purple are the FPGA-based IP cores from Xilinx, and the blocks in green are designed and

implemented by us.

FPGA

Classification FIFO1

OPS Label Gen10Gbps GTH Transceiver

From Control Plane

LUT/Control

Aggregate

ControlOPS/OCS

10G MAC

OCS Tx Buffer

10Gbps GTH Transmitter


...

OPS Label Gen

OPS Tx Buffer

Dest MAC, Src MAC, VLAN ID



Seggregation/Processing

OCS/OPS

Switch

10Gbps GTH Receiver

10Gbps GTH Receiver

10Gbps GTH Receiver

10Gbps GTH Receiver




OPS/OCS Hybrid

OCS

OPS/OCS Hybrid

OCS

RX FIFO

RX FIFO

RX FIFO

RX FIFO

Figure 64 FPGA-based OPS/OCS interface design architecture

The Xilinx Virtex-6 FPGA GTH transceiver is highly configurable and tightly integrated with the

programmable logic resources of the FPGA. The FPGA TX interface is the FPGA’s gateway to the

TX datapath of the GTH transceiver. Applications transmit data throught the GTH transceiver by

writing data to the TXDATA port on the positive edge of TXUSERCLKIN. Similar the transmitter

side, the GTH receives the data by reading the RXDATA port on the positive edge of

RXUSERCLKIN. One critical functional block of RX is the RX clock data recovery (CDR) circuit

in each GTH transceiver, which extracts the recovered clock and data from an incoming data stream.

This affects the limited bandwidth when sending/receiving OPS traffic because it requires to leave a

gap between two adjacent packets for the GTH receivers to recover the clocks.



of 78

The Xilinx IP 10-Gigabit Ethernet MAC core is a single-speed, full-duplex 10Gbps Ethernet Media

Access Controller solution enabling the design of high-speed Ethernet systems and subsystems. It is

designed to 10-Gigabit Ethernet specification IEEE 802.3-2008 and supports the high-bandwidth

demands.

Multiple First-in-first-out (FIFO) and buffers are used for synchronization between different clock

domains to avoid any metastability problems occurred in the FPGA. A FIFO is also employed in the

OPS route to synchronize sending the packets. When dealing a queue in the receiver side, the

priority mechanism is used which means when the traffic enters the FPGA-based interface at the

same time in port1, 2, 3 and 4, the traffic from port1 first goes through, then traffic from port2, then

port3, lastly port4. Port1 always has the priority than port2, and so on.

The classification functional blocks (demonstrated in Figure 65a) are designed and implemented to

classify the traffic based on the Ethernet header which includes destination MAC address, source

MAC address, and VLAN ID. We used 4 classification FIFOs, the matching mechanism is shown in

Figure 65b. The matching is bit-based, and 6 bytes for destination MAC address, 6 bytes for source

MAC address and 4 bytes for VLAN ID. The mask (‘1’ means select, ‘0’ means not) works on a bit-

based map and selects the bits of header to match the traffic. In the example, the mask of buffer1 has

a “F” in the last byte of destination MAC address, and a “1” in the late byte of VLAN , it means

when the traffic with Destination MAC address XX.XX.XX.XX.XX.X6 and VLAN group of even

numbers (0,2,4…) will be put in buffer 1. (“X” means don’t care)


Dest MAC, Src MAC, VLAN ID




a) b)

Figure 65 a) classification functional blocks; b) traffic flow match table

The FPGA-based OPS/OCS interface is able to communicate with the control plane through

dedicated 10Gbps interface. The FPGA-based design and implementation interface is shown in

Figure 66. The system employs a server-based control agent for translating the commands from

control plane to the FPGA-based OPS/OCS interface, and vice versa. The control agent sends the

commands capsuled in a pre-defined 1508Bytes Ethernet frame, and FPGA-based OPS/OCS

interface is capable of extracting the commands from this pre-defined Ethernet frame. Meanwhile,

FPGA-based OPS/OCS interface updates control agent its status by sending the same pre-defined

1508Bytes Ethernet frame. The commands and status information are stored in a Look-up-table

(LUT) deployed in the FPGA-based interface.



of 78

OPS Label Gen10Gbps GTH Transceiver

From Control Plane

LUT/Control

10G MAC

...

OPS Label Gen

Figure 66 FPGA-based functional blocks for control agent interface

There are 4 ports designed and implemented, 2 as hybrid ports sending/receiving either OCS or OPS

traffic, and 2 as OCS-only ports which can only send and receive OCS traffic. When the receiving

end of the FPGA-based OPS/OCS interface receives the traffic, firstly, they will be segregated (OPS

traffic) or processed and identified the header and tailor (OCS traffic), and then the processed data

will be classified based on Destination MAC address, Source MAC address and VLAN ID into

separate FIFOs. Finally, according the control plane’s commands, the data will be organized as OCS

or OPS format and then transmitted out through dedicated port.

The FPGA-based design and implementation described above are completed. The functions are

tested and proven on board. We will continue with more experiments using the FPGA-based

OPS/OCS interface in the system for further debugging and results.

6 Conclusions

Based on the requirement analysis of the technology enablers for the flexi-grid optical path-packet

infrastructure for the Ethernet transport, as reported in D2.2, a detailed investigation and assessment

for the potential technologies is done by experiments and simulations.

The advantage of OFMD technologies in optical shot-reach transmission systems has been

validated in the 400Gb/s optical Ethernet scenario. A 400Gb/s DMT transponder prototype has

been implemented and up-to 40km error-free operation with 7% FEC has been demonstrated.

FL-VC OPS network based on the OFDM technologies have been investigated and some

practical adaptive-modulation technique, such as for designing the cross-talk tolerant packets,

have been proposed. Furthermore, application throughput and latency of the distance-adaptive

DMT-based OPS network have been analyzed.

For the implementation of the sliceable OFDM transceiver, an OFDM-based BVT, able to

generate a single multi-format rate/distance adaptive flow, has been studied and optimized. It

can be used as S-BVT building block for cost-sensitive applications, targeting the

metro/regional network segment and inter-DC communication. Rate/margin adaptive

algorithms have been developed and integrated within the DSP modules of the BVT to enable

bit and power loading of the individual subcarriers for rate/distance adaptive transmission

according to the capacity request and channel estimation. The optoelectronic front-end at the

receiver uses direct detection (DD) and it is combined with single sideband (SSB) modulation

at the bandwidth variable transmitter. Two SSB-OFDM schemes have been compared for

different spectral efficiency/occupancy, exploiting the adaptive multi-format assignment. No

guard-interval SSB is proposed to be used considering DAC limitation and transmission

impairments. We have analysed the system performance by means of numerical simulations

and we have experimentally assessed the BVT including the implemented adaptive DSP

modules.



of 78

A QPSK/16QAM multi-format transmitter is setup with baud rate up to 40Gbaud,

carrying signals up to 320Gbit/s (40Gbaud PM-16QAM) for spectral efficiency

transmission. The designed transmitter is feed traffic by our designed integrated

OPS/OCS interfaces, and provide further optical aggregation for Ethernet traffic from

OPS domain. The multi-format transmitter adopts different modulation format for

optical links with variable distance to optimize transmission performance. At the

receiver side, we develop a full set of digital signal processing (DSP) modules to

mitigate fiber impairments and the carrier frequency offset (CFO) for intradyne

detection. The DSP algorithms can handle both QPSK and 16QAM signals for variable

baudrate.

With regards to optical node design, flexible/adaptable optical nodes is developed based on

AoD concept for flexi-grid DWDM networks. The node composing algorithms are used to

synthesis and reconfigure the optical nodes, while the optical monitoring modules monitor the

synthesized optical node and guarantee it work correctly. The proposed AoD-based optical

nodes deploy network functions according to the traffic requests, which will improve hardware

utilization efficiency. For the prototype implementation, we have achieve several optical node

functions, such as architecture programmable ROADM, software-defined programmable

transmitter. The software-defined programmable transmitters provide interfacing between OPS

and flexi-grid OCS domain. The proposed flexible optical node shows a reduced the cost and a

lower power consumption, comparing with other ROADM architectures.

The OPS-OCS integrated interface is developed with FPGA develop board (HTG-V6HXT-

X16PCIE board), which provide two hybrid ports (OCS/OPS traffic) and two OCS-only ports.

The hybrid ports are used to connect to the OPS domain, while the OCS-only ports are to the

OCS domain. The OPS-OCS integrated interface convert OPS traffic to OCS domain and

forward traffic to high baud rate transmitters in OCS domain for spectral efficiency

transmission.

7 List of acronyms

ADC Analog-to-Digital Converter

AoD Architecture-on-Demand

BGP-LS Border Gateway Protocol- Link State

BRAS Broadband Remote Access Server

BVT Bandwidth-Variable Transponders

BV-OXC Bandwidth Variable – Optical Cross-Connect

BV-

ROADM

Bandwidth Variable – Reconfigurable Optical Add-Drop Multiplexer

CAPEX Capital Expenditures



of 78

CD/C Colorless Directionless Contentionless

CFO Carrier Frequency Offset

COP Control Orchestration Protocol

CRDL Common Resource Description Language

DAC Digital-to-Analog Converter

DCN Data Center Network

DDoS Distributed Denial-of-Service

DMT Discrete Multi-Tone

ECL External Cavity Laser

E2E End-to-End

FDL Fiber Delay-Line

FEC Forward Error Correction

FPGA Field Programmable Gate Array

GMPLS Generalized Multiprotocol Label Switching

LPFS Large Port Count Fiber Switch

LUT Look-Up-Table

MEMS Micro-Electro-Mechanical System

IX Internet eXchange

NE Network Element

NVC

OCS

Network Virtualization Controller

Optical Circuit Switching

OF OpenFlow

OFDM Orthogonal Frequency Division Multiplexing



of 78

OFS OpenFlow switch

ONF Open Networking Foundation

ONH Optical Network Hypervisor

OOK On-Off Keying

OPEX Operational Expenditures

OPS Optical Packet Switching

PAPR Peak-to-Average Power Ratio

PCE Path Computation Element

PM Polarization Multiplex

QoS Quality of Service

SC Superchannel

SC-FDM Single Carrier Frequency Division Multiplexing

SDN Software Defined Networking

SNR Signal-to-Noise Ratio

SSMF Standard Single Mode Fiber

SSS Spectrum Selective Switch

TED Traffic Engineering Database

TDM Time Division Multiplexing

VC Virtualization Composer

VOA Variable Optical Attenuator

VON Virtual Overlay Network

VP Virtualization Partitioner

WDM Wavelength Division Multiplexing



of 78

XML Extensible Markup Language



of 78

8 References

All the reference material and the list of acronyms have to be listed in alphabetic order.

[Agrawal07] N. Agrawal, W. Bolosky, J. Douceur, J. Lorch, “A five-year study of file-system

metadata,” ACM Transactions on Storage, vol. 3, no. 3, aa. 9, Oct. 2007.

[Agrawal10] G. Agrawal, ``Fiber-Optic Communication Systems,’’ Wiley, 4th. Ed., pp. 50—

54, 2010.

[AkamaiC] Akamai Caching Service,

https://developer.akamai.com/stuff/Caching/Content_Caching.html.

[Alizadeh10] M. Alizadeh, A. Greenberg, D.A. Maltz, J. Padhye, P. Patel, B. Prabhakar, S.

Sengupta, M. Sridharan, “Data Center TCP (DCTCP),” ACM SIGCOMM 2010,

New Delhi (India), Aug.-Sept. 2010.

[Auge13] J.-L. Auge, “Can we use flexible transponders to reduce margins?,” in Optical

Fiber Communication Conference and Exposition and the National Fiber Optic

Engineers Conference (OFC/NFOEC), 2013, 2013, pp. 1–3.

[Barros10] D.J.F. Barros and J.M. Kahn, “Comparison of Orthogonal Frequency-division

Multiplexing and On-off Keying in Amplified Direct-detection Single-mode Fiber

Systems,” IEEE/OSA J. Lightwave Technol., vol. 28, no. 12, pp. 1811–1820,

2010.

[CAIDA08] Center for Applied Internet Data Analysis (CAIDA), ``Packet size distribution

comparison between Internet links in 1998 and 2008,

http://www.caida.org/research/traffic-analysis/pkt_size_distribution/graphs.xml

[Cioffi] J. Cioffi, “Data transmission theory, chapter 4 of course text for EE379A-B and

EE479,” Stanford University. [Online]. Available:

www.stanford.edu/group/cioffi/book/

[Clark14] D. Clark, S. Bauer, W. Lehr, K.C. Claffy, A. Dhamdhere, B. Huffaker, M. Luckie,

``Measurement and Analysis of Internet Interconnection and Congestion,’’

Telecommunications Policy Research Conference (TPRC), Arlington, VA (USA),

Sept. 2014.

[GoogleGC] Google Global Cache, https://peering.google.com/about/ggc.html.

[Gnauck09] A.H. Gnauck, etc. “10 × 112-Gb/s PDM 16-QAM Transmission Over 630 km of

Fiber with 6.2-b/s/Hz Spectral Efficiency”, OFC2009, PDPB8.

[Gnauck10] A.H. Gnauck, etc. “10*224Gb/s WDM transmission of 28-GBaud PDM 16-QAM

on a 50-GHz Grid over 1200 KM of fiber”, OFC2010, PDPB8.

[Greenberg09] A. Greenberg, J.R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D.A. Maltz,

P. Patel, S. Sengupta, ``VL2: A Scalable and Flexible Data Center Network,’’

ACM SIGCOMM 2009, Barcelona (Spain), Aug. 2009.

[Gringeri10] S. Gringeri, B. Basch, V. Shukla, R. Egorov, and T. J. Xia, “Flexible architectures

for optical transport nodes and networks,” Commun. Mag. IEEE, vol. 48, no. 7,

pp. 40–50, 2010.

[Isono14] Hideki Isono, Tomoo Takahara, Hisaya Sakamoto, Yuji Miyaki, and Toshiki

Tanaka: Production feasibility study on 400GbE PMD for SMF objectives, in

IEEE802.3bs 400G Ethernet Study Group San Antonio Plenary, Nov.

[Ji10] P. N. Ji and Y. Aono, “Colorless and directionless multi-degree reconfigurable

optical add/drop multiplexers,” in Wireless and Optical Communications

Conference (WOCC), 2010 19th Annual, 2010, pp. 1–5.

[Kitayama15] Ken-Ichi Kitayama, et al., “Torus-Topology Data Center Network Based on

Optical Packet/Agile Circuit Switching with Intelligent Flow Management,” J.



of 78

Lightwave Technol., Vol. 33, no. 5, pp. 1063-1071, 2015.

[Kobayashi12-

1]

T. Kobayashi, A. Sano, A. Matsuura, Y. Miyamoto, and K. Ishihara, “High-Order

QAM Transmission for Spectrally-efficient and High-capacity Transport,” in

Optical Fiber Communication Conference, 2012, p. OM2A.3.

[Kobayashi12-

2]

T. Kobayashi, A. Sano, A. Matsuura, Y. Miyamoto, and K. Ishihara, “Nonlinear

tolerant long-haul WDM transmission over 1200km using 538Gb/s/ch PDM-

64QAM SC-FDM signals with pilot tone,” in Optical Fiber Communication

Conference, 2012, p. OM2A.5.

[Li13] Y. Li, Y. Zhang, and Y. Huang: “Any bias control technique for Mach-Zehnder

modulator,” IEEE Photon. Technol. Lett., vol. 25, no. 24, pp. 2412-2415, Dec.

2013.

[Liu12] X. Liu, S. Chandrasekhar, P. Winzer, T. Lotz, J. Carlson, J. Yang, G. Cheren, and

S. Zederbaum, “1.5-Tb/s Guard-Banded Superchannel Transmission over 56x

100-km (5600-km) ULAF Using 30-Gbaud Pilot-Free OFDM-16QAM Signals

with 5.75-b/s/Hz Net Spectral Efficiency,” in European Conference and

Exhibition on Optical Communication, 2012, p. Th.3.C.5.

[Losada14] P.J. Argibay-Losada, K. Nozhnina, G. Sahin, C. Qiao, ``Using stop-and-wait to

improve TCP throughput in fast optical switching (FOS) networks over short

physical distances,’’ IEEE INFOCOM 2014, Toronto, ON (Canada), Apr.-May

2014.

[Mathis96] M. Mathis, J. Mahdavi, S. Floyd, A. Romanow, ``TCP Selective Acknowledgment

Options,’’ IETF RFC 2018, Oct. 1996.

[MS6] “Analysis and evaluation of design proposals for transceivers, optical nodes, and

interfaces in the flexi-grid optical path-packet infrastructure for Ethernet

transport”

[Nadal14] L. Nadal et al, “DMT modulation with adaptive loading for high bit rate

transmission over directly detected optical channels,” J. Lightwave Technol., 32,

21, p.4143 (2014).

[Peng12] W.-R. Peng, H. Takahashi, T. Tsuritani, and M. Itsuro, “DAC-free generation and

1200-km Transmission of 41-Gbd PDM-64QAM using a single I/Q modualator,”

presented at the Opto-Electronics and Communications Conference (OECC 2012),

Busan, Korea, 2012, vol. PDP1–3.

[Qian14] Y. Qian, H. Mehrvar, H. Ma, X. Yang, H.Y. Fu, K. Zhu, D. Geng, D. Goodwill, P.

Dumais, and E. Bernier, “Crosstalk optimization in low extinction-ratio switch

fabrics,” in Proc. IEEE/OSA OFC 2014, Paper Th1I.4, 2014.

[Ramamurti01] V. Ramamurti and G. Young, Ethernet Transport over RPR. IEEE, 2001.

[Schmidt08] B. Schmidt, A. Lowery, and J. Armstrong, “Experimental demonstrations of

electronic dispersion compensation for long-haul transmission using direct-

detection optical OFDM,” J. of Lightw. Technol., vol. 26, no. 1, pp. 196–203,

2008.

[Shi11] Y. Shi, Y. T. Hou, S. Kompella, and H.D. Sherali, “Maximizing capacity in

multihop cognitive radio networks under the SINR model,” IEEE Trans. Mobile

Comput., vol. 10, no. 7, pp. 954-967, July 2011.

[Shinada14] S, J.M. Delgado-Mendinueta, R. Luís, N. Wada, “Operation of a 12.8 Tbit/s

DWDM Polarization Division Multiplexing 16-QAM Optical Packet Switching

Nodeafter 50-km of Fiber Transmission,” in Proc. ECOC2014, Paper We.3.5.4.,

Sept. 2014.

[Stassar14] Peter Stassar and Chris Cole: SMF PMD Decision Tree Status, in IEEE802.3bs 400G

Ethernet Study Group Ottawa Interim, Sep. 2014.

[Svaluto12] M. Svaluto Moreolo, et al.: Experimental demonstration of a cost-effective bit rate



of 78

variable IM/DD optical OFDM with reduced guard band, Opt. Express 20, B159-

B164 (2012).

[Svaluto15] M. Svaluto Moreolo, L. Nadal, J.M. Fabrega, “DSP-enabled Optical OFDM for

Multiple-format and Multi-rate/distance Transmission, ” invited paper ICTON

2015, 5-9 July 2015, Budapest, Hungary.

[Takahashi12] R. Takahashi, T. Nakahara, T. Segawa, H. Ishikawa, S. Ibrahim, and Y. Suzaki,

“Recent progress on hybrid optoelectronic router for future energy-efficient

optical packet switched networks,” Proc. Photonics Switching (PS2012), Corsica,

France, Sep. 2012.

[Takahashi14] R. Takahashi , T. Nakahara , K. Takahata , H. Takenouchi , T. Yasui , N. Kondo

and H. Suzuki “Ultrafast optoelectronic packet processing for asynchronous,

optical-packet-switched networks”, J. Opt. Netw., vol. 3, no. 12, pp.914 -930,

2004.

[Tanaka14] Toshiki Tanaka, Tomoo Takahara, Hideki Isono, Hisaya Sakamoto and Yuji

Miyaki: Technical Discussion on 400GbE PMD for SMF Objectives, in

IEEE802.3bs 400G Ethernet Study Group San Antonio Plenary, Nov. 2014.

[Walker] Timothy P. Walker, “Optical Transport Network (OTN) Tutorial.”

[Web14] “The Way to TbE : Media & Entertainment Technology.” 2014,

http://mandetech.com/2014/05/13/the-way-to-tbe-2/

[WIDE15] WIDE, ``Packet Traces from WIDE backbone,’’ \url{mawi.wide.ad.jp/mawi}

[Yokokawa11] M.Yokokawa, F.Shoji, A.Uno, M.Kurokawa, and T.Watanabe, “The K computer:

Japanese next-generation supercomputer development project,” Proc. IEEE/ACM

Int. Symp. Low-Power Electron. Design, pp. 371–372, 2011.

[Yoshida14] Y. Yoshida, T. Kodama, S. Shinada, N. Wada, and K. Kitayama, “Fixed-length

Elastic-capacity OFDM Payload Packet: Concept and Demonstration” in Proc.

IEEE/OSA OFC2013, Paper Out2A.7., 2013

[Yoshida14] Y. Yoshida, A. Maruta, K. Kitayama, M. Nishihara, T. Takahara, T. Tanaka, J.

Rasmussen, N. Yoshikane, T. Tsuritani, I. Morita, S. Yan, Y. Shu, Y. Yan, R.

Nejabati, G. Zervas, D. Simeonidou, R. Vilalta, R. Munoz, R. Casellas, R.

Martinez, V. Lopez, A. Aguado, and J. Beltran, “SDN-based Network

Orchestration of Variable-capacity Optical Packet Switching Network over

Programmable Flexi-grid Elastic Optical Path Network,” J. Light. Technol., vol.

33, no. 3, pp. 609–617, Aug. 2014.

[Zhou12] Xiang Zhou, L. E. Nelson, P. Magill, R. Isaac, Benyuan Zhu, D. W. Peckham, P.

I. Borel, and K. Carlson, “PDM-Nyquist-32QAM for 450-Gb/s Per-Channel

WDM Transmission on the 50 GHz ITU-T Grid,” J. Light. Technol., vol. 30, no.

4, pp. 553–559, Feb. 2012.

deliverable d2.2 investigation and assessment of the ... d2.2.final.pdf · for the flexi-grid...

Documents