Optical Networking Interconnecting Disaggregated Compute Resources: An
enabler of the 5G Vision
AnnaTzanakaki(1),(2),Markos P.Anastasopoulos(1),DimitraSimeonidou(1)
(1) UniversityofBristol,UK(2)NationalandKapodistrian UniversityofAthens,DepartmentofPhysics,Greece
Outline
§ Motivation 5G vision
§ Proposed 5G architecture
§ Optical Networking Interconnecting Compute Resources
Inter- and Intra-DC connectivity
Disaggregation of Resources
Increase granularity in the control of resources and utilisation efficiency
§ Modelling results: power consumption
§ Conclusions
1
Motivation - 5G Vision• Newbusinessopportunitiesmeetingtherequirementsoflargevarietyofusecases
(enablingverticals)• 5GNetworkstobefutureproof(i.e.designtoevolveandchangeandnottobereplaced)
• bymeansofimplementingnetworkslicinginacostefficientway
– addressingbothenduserandoperationalservices
– supportingnativesoftwarization
– integratingcommunicationandcomputation
– integratingheterogeneoustechnologies(incl.fixedandwirelesstechnologies)
5G PPP View on 5G Architecture – (White Paper), https://5g-
ppp.eu/white-papers/
2
Supporting operational and end-user services
EPCMME/SGSN
P-SGW/GSN InternetMacro-cell
IndoorWireless
OpticalTransport
WDM-PON
SmallCells
LTE
BBUBBU
BBUBBU
BBUBBU
BBUBBU
MobileFronthaul MobileBackhaul
DataCenters
RRHRRHRRH
RRHRRHRRH
VM
vBBUvBBU Internet
Fronthaul
OpticalTransportDataCenters
BBU
RRHRRHRRH
BackhaulWirelessAccess
WiFi
SmallCells
• Early solutions: Separate Fronthaul and Backhaul networks• Fronthaul: Network between RRHs and dedicated BBUs
• Backhaul: Between the core network and the sub-networks at the edge
• C-RAN: Sharing of BBUs but Separate Fronthaul and Backhaul networks• Proposal: Adopt C-RAN and offer a common transport to support jointly
converged FH and BH functionalities.• high transport bandwidth
• Strict delay constraints• enhanced synchronisation capabilities
• flexibility and elasticity in bandwidth allocationA. Tzanakaki et al., Wireless-Optical Network Convergence: Enabling the 5G Architecture to Support Operational and End-User Services, IEEE Commun. Mag. 2017 (in press) 3
Flexible Functional Splits
RFtoBaseband
CyclePrefix&FFT
Resourcedemapping
Receiveprocessing
Decoding
MAC
(1)
(2)
(3)
(4)
(5)
Clou
dRA
NTrad
ition
alR
AN
• highnetworkbandwidth• IncreasedBBUsharing
• lownetworkbandwidth• LimitedBBUsharing
• FlexiblesplitprocessingtorelaxthestringentrequirementsofFHservicesfor:– transportcapacity– delay– synchronisation
• SplitOptions– Split(1):“CPRI”– Split(2):“load-dependentFH”– Split(5):“BHwithcentralizedscheduling”– Processingcanbealsoperformedatgeneralpurposeservers
A. Tzanakaki et al., "5G infrastructures supporting end-user and operational services: The 5G-XHaul architectural perspective," IEEE ICC, 2016.
4
Overall Infrastructure
• Key Challenge: • Converged Fronthaul and Backhaul Services• Converged Network and Compute Services• Disaggregation of Resources
EdgeNode
RFtoBaseband
CyclePrefix&FFT
Resourcedemapping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)
• highnetworkba
ndwidth
• IncreasedBB
Usharing
• lownetworkband
width
• Limite
dBB
Ush
aring
(2)
(1)
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUXBV-WSS
BV-WSS
BV-WSS
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUX
b)
c)
RU:RemoteUnit
ProcessingattheServerRU
ProcessingattheRU
BVT:BandwidthvariabletransponderBV-WSS:Bandwidthvariablewavelengthselectionswitch
CoreNode
OpticalLink
a)
RU1
RU3 Core
Node
CoreNode
EdgeNode
EdgeNode
SPP
SPP
GPP
GPP
ToR
ToR
BVT
λ1
λ2
λΝ
EdgeNode
SPPSPPSPP
SPP
ToR
SPP
CPUCPU
SPP
ToR ToR
RU2
(5)
(2)
ElasticOpticalNetwork DataCentre
Tx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
BVT
BVT
λ1
λ2
λΝ
λ1
λ2
λΝ
SFPETHTx
I/QTxRU Sw
itch
SYNC
BV-WSS
Multi-protocolInterfa
ce
SwitchSYNC
Multi-protocolInterface
(1)CPUSPP SPP SPP
Switch Switch
(3) (4)
d
RU2
SPP
SPP
GPP
GPP
SPP
SPP
GPP
GPP
GPP:GeneralPurposeProcessorSPP:SpecificPurposeProcessor
FronthaulFlow
5
Physical Network Infrastructure
• Integratedopticalandwirelessnetwork infrastructurefortransportandaccess• Wirelessdomain:
– Denselayerofsmallcellscomplementedbymacrocellstoensure ubiquitouscoverage– Smallcellscan bebackhauledtothemacro-cellsiteeitherwirelessly usingacombinationofmm-
Waveandsub-6wirelesstechnologiesorusingahybridopticalnetworkplatform• OpticalDomain
– Adoptionofadynamicandflexible/elasticframebasedopticalnetworksolutioncombinedwithenhancedcapacityWDMPONs
• BBProcessing:RUsareconnectedtoremoteBB processing pools throughhighbandwidthtransportlinks
VM
vBBU2
DataCenters
SmallCells
RRH
RRH
RRH
RRH
eNB
RRH
RRHRRHRRH
EPCSGW/GSN
vBBU
vBBUvBBU
PDN-GW
vBBU1
Backhaul
Fronthaul
vBBU
vBBU1 vBBU2
VM
eNB
MacroCell
Internet
60GHz/Sub-6linksforFH/BHFHBH
HeNBFemtoCells
HeNBHeNBGW
HeNB
WiFiWLANGW
TSON
WDM-PON
5G-XhaulWirelessBackhaul/Fronthaul
WirelessAccess
EdgeNode
RFtoBaseband
CyclePrefix&FFT
Resourcedemapping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)
• highnetworkba
ndwidth
• IncreasedBB
Usharing
• lownetworkband
width
• Limite
dBB
Ush
aring
(2)
(1)
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUXBV-WSS
BV-WSS
BV-WSS
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUX
b)
c)
RU:RemoteUnit
ProcessingattheServerRU
ProcessingattheRU
BVT:BandwidthvariabletransponderBV-WSS:Bandwidthvariablewavelengthselectionswitch
CoreNode
OpticalLink
a)
RU1
RU3 Core
Node
CoreNode
EdgeNode
EdgeNode
SPP
SPP
GPP
GPP
ToR
ToR
BVT
λ1
λ2
λΝ
EdgeNode
SPPSPPSPP
SPP
ToR
SPP
CPUCPU
SPP
ToR ToR
RU2
(5)
(2)
ElasticOpticalNetwork DataCentre
Tx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
BVT
BVT
λ1
λ2
λΝ
λ1
λ2
λΝ
SFPETHTx
I/QTxRU Sw
itch
SYNC
BV-WSS
Multi-protocolInterfa
ce
SwitchSYNC
Multi-protocolInterface
(1)CPUSPP SPP SPP
Switch Switch
(3) (4)
d
RU2
SPP
SPP
GPP
GPP
SPP
SPP
GPP
GPP
GPP:GeneralPurposeProcessorSPP:SpecificPurposeProcessor
FronthaulFlow
6
Elastic Optical Network
• Opticalnetworkelasticityinthefrequencydomain
• Addressesefficientlythegreatlyvaryingbandwidthrequirementsofthevarious5Gservices– E.g.heavyCPRIflows,lightersplit
options,enduserservices
serverETHTx
I/QTx
serverETHTx
I/QTx
serverETHTx
I/QTx
serverETHTx
I/QTx
serverETHTx
I/QTx
serverETHTx
I/QTx
EthernetSwitch
BVR
BVR
Tx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
SFP RUETHTx
I/QTx
EthernetSwitch BVT
EdgeNode
BVT
λ1
λ2
λΝ
λ1
λ2
λΝ
Spectrum
HeavyCPRI
LightCPRI
LightEthernet
HeavyEthernet
Wastedresources
HeavyCPRI
LightCPRI
LightEthernet
HeavyEthernet
Spectrumsavings
Spectrumsavings
Spectrum
I) II)
III) IV)
SFPETHTx
I/QTxRU
ACGroup1{AC0,AC1}
ACGroup2{AC1,AC2}
BV-WSS
BV-WSS
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUX
BV-WSS
BV-WSS
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUX
I/QSwitch
SYNC
I/QSwitch
SYNC
c)
d)
7
Convergence of Networking and Computation
MegaDCs Internet
Metro
Core
Metro
RegionalDCs
Mid-DCs
Micro-DCs
WirelessAccessMacro-cells
ControllerResources
NFVI
VIM
ControllerResources
NFVI
ControllerResources
NFVI
ControllerResources
NFVI
VIM
VIM
VIM
VNF
VNF
SSD
RAMGPP SPPPNF
PNF
VNF SSD
RAMGPP SPP
PNF
PNF
VNF
VNF
VNF
Frontlhaul
VNF
LocalDCsWireless
VirtualizationPHYFunctions
RegionalDCsOpticalMetro RemoteDCsOpticalCore
VirtualizationPHYFunctions VirtualizationPHYFunctions
PHYInf
VirtualResources
VirtualResources
VirtualResources
PHYResources
PHYResources
PHYResources
PNF VNFPNF VNF VNF VNF
VirtualTransport
forwarding
vBBU(lowerlayers)
vBBU(upperlayers)
EndpointEndpoint
forwardingforwarding
OperationalNetworkServices
End-userservices
RFto
Baseb
and
Cycle
Prefix&FFT
Resource
demapping
Receive
processin
g
Decoding
MAC
Service
Man
agem
ent&
Orche
stratio
nInf.Man
agem
ent&
Co
ntrol
LocalDCs Remote/regionalDC
LocalDCsWireless
VirtualizationPHYFunctions
RegionalDCsOpticalMetro RemoteDCsOpticalCore
VirtualizationPHYFunctions VirtualizationPHYFunctions
PHYInf
VirtualResources
VirtualResources
VirtualResources
PHYResources
PHYResources
PHYResources
PNF VNFPNF VNF VNF VNF
VirtualTransport
forwarding
vBBU(lowerlayers)
vBBU(upperlayers)
EndpointEndpoint
forwardingforwarding
OperationalNetworkServices
End-userservices
RFto
Baseb
and
Cycle
Prefix&FFT
Resource
demapping
Receive
processin
g
Decoding
MAC
Service
Man
agem
ent&
Orche
stratio
nInf.Man
agem
ent&
Co
ntrol
LocalDCs Remote/regionalDCHWA GPP
• Fronthaul adoptingflexiblefunctionalsplits
• TransportNetworkandComputeResources
• Processingparallelisation
• FullGPPwithcommodityhardwareandhybridsolutions
• DisaggregationofNetworkandComputeresources
• Mixingandmatchingofresourcestoefficientlysupportservices
8
Processing Parallelization
SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)
CoreNodeEdge
NodeRU2
EdgeNode
SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)
d)
(3)
c)b)a)
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU DCOpticalinterDCnetwork
Transportrequirements:Split2
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU SPP/GPPIntraDCnetwork
Transportrequirements:
Split3
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU SPP/GPPIntraDCnetworkTransport
requirements:Split4
SPP/GPPSPP/GPP
• BBUsub-frameprocessingtime<1ms• Toreduceprocessingdelay,BBprocessingishandledinparallelover
disaggregatedcomputeresources• Model1:parallel(orsequential)processingmode
– eachfunctionisdistributedacrossmultipleprocessingunits(1:N)
• Model2:Pipelining– eachprocessingunithandlesaspecificfunctionadopting1:1mapping
9
Use case: Joint optimization of Inter-intra DC network for FH services
Sub-problem1:TransportNetworkOptimization• Identificationofoptimalresourcestosupport
FHorBHservicesintermsofbothtopologyandresourcesincludes:– Ordering(SC) oftherelevantfunctions(VNFor
PNF)– Estimatingvirtualresourcesrequiredtosupport
SCandexecuting thecorresponding applications– Mapping virtualresourcestophysicalresources
MegaDCsInternet
Metro
Core
Metro
MobileEdgeDCs
RegionalDCs
WirelessAccessMacro-cells
SSD
RAMGPP SPP
SSD
RAMGPP SPP
SSD
RAMGPP GPP
SSD
RAMGPP SPP
SSD
RAMGPP SPP
SSD
RAMGPP SPP
SSD
RAMGPP SPP
SSD
RAMGPP SPP
RegionalDCs
Disaggregatedresources• RAM,GPP,SPP• HWAccelerators
M. P. Anastasopoulos, A. Tzanakaki and D. Simeonidou, "Energy-aware offloading in mobile cloud systems with delay considerations," 2014 IEEE Globecom Workshops (GC Wkshps), Austin, TX, 2014, pp. 42-47.
10
Sub-problem 1: Constraints
• BBUchaining:VNFsareprocessedintheorderdefinedbythecorrespondingSC
• Flowconservationaswellasmappingandaggregation/de-aggregationoftrafficbetweendifferentdomains
• End-to-enddelayguarantees• FlexibleFHprocessingsplits
– BBUProcessingconstraints– RUDemandConstraints
• Transportnetworkcapacityrequirements– Constraintsimposedbytheelastic
opticalnetwork
LargeDCs Internet
Metro
Core
Metro
MiniDCs
Mid-DCs
Micro-DCs
WirelessAccessMacro-cells
ControllerResources
NFVI
VIM
ControllerResources
NFVI
ControllerResources
NFVI
ControllerResources
NFVI
VIM
VIM
VIM
Frontlhaul
VNF
VNF
SSD
RAMGPP SPPPNF
PNF
VNF SSD
RAMGPP SPP
PNF
PNF
VNF
VNF
VNFBackhaul
vBBU vBBU
Frontlhaul
11
Sub-problem 2: Disaggregated DC Network Optimization
Sub-problem2:• IdentifiestheoptimalprocessingmoduleswherepartsoftheFHSChavetobe
allocated.• EstablishmentofapathinterconnectingedgeDCnodewiththeGPP/SPP
modulesthatwillprocesstheremainingFHfunctions• TheorderofFHfunctionsprocessingisdefinedbythecorrespondingSC
SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)
CoreNodeEdge
NodeRU2
EdgeNode
SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)(3)SPP SPP
ToR
(5)SPP SPP
Switch Switch
(4)
d)
(3)
c)b)a)
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU DCOpticalinterDCnetwork
Transportrequirements:Split2
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU SPP/GPPIntraDCnetwork
Transportrequirements:
Split3
RFtoBaseband
CyclePrefix&FFT
Resourcedem
apping
Receiveprocessing
Decoding
MAC
(3)
(4)
(5)(2)
(1)
RU SPP/GPPIntraDCnetworkTransport
requirements:Split4
SPP/GPPSPP/GPP
Sub-problem 1
12
min𝐹% =' ℰ) ' 𝑝)+,-𝜎)+�
+12+' ℰ4 𝐶4
4∈𝐄)∈𝐑
Optimization framework• JointoptimizationofInter/IntraDCNetwork• 1st Sub-problem:Optimaltransportnetworkisidentifiedthroughthe
minimizationoftheenergycost:
subjecttocapacity,latencyanddemandconstraints.• 2nd Sub-problem:DisaggregatedDCNetworkOptimization
subjecttodemandprocessingandcapacityconstraints
FH energy, local processing FH energy, transport
min𝐹9 =' ℰ: ' ' ' 𝑝;𝑎;=�
;>𝐅𝐇𝒓𝒊𝒅+12)∈𝐑
+' ℰ4 𝐶44∈𝐄𝒅:∈𝐌𝒅
FH energy, DC processing FH energy, DC network
13
Numerical evaluation: Scenario
• BristolisOpen(BIO)smartcitytest-bedtopology• Snapshotofspatialtrafficloadandaveragetraffic/BSbasedonrealdatasets
0
5
10
0
5
10
20
40
60
80
Km
Aver
age
data
rate
(Mbp
s) 0 20 40 60 80
X (km)Y (km)
OpticalEdgeNodeDataCenterOpticalLink
14
Numerical results: Energy
• “Distributed-RAN(D-RAN)”RUsandBBUsareco-locatedandFHserviceprocessingiscarriedoutexclusivelybyspecificpurposehardware.
• “C-RANwithspecificpurpose(SP)BBUhardware(SP-C-RAN)”:SpecificpurposeBBUsareplacedatacentralizedlocationtobeshared.
• “C-RANwithvirtualBBUs(V-C-RAN)-”:AllowsBBUstobeinstantiatedasvirtualfunctionsandrunonGPPSenablingsharingofresourcesandon-demandresizingofcomputeresources.
• “Disaggregated-RAN(DA-RAN)”:CombinesthebenefitsofSP-C-RANandV-C-RANallowingFHfunctionstobeprocessedeitheratSPPorGPPbasedontheirspecificcharacteristics.
15
Conclusions
§ Motivation 5G vision
§ Proposed 5G architecture
§ C-RAN
§ Flexible Functional Splits
§ Optical Networking Interconnecting Compute Resources enablingResource Disaggregation
Increased resource efficiency
§ Modelling results indicate significant benefits in power efficiencycompared to alternative approaches
17
Sub-problem 1: Constraints
• BBUchaining:VNFsareprocessedintheorderdefinedbythecorrespondingSC
• Flowconservationaswellasmappingandaggregation/de-aggregationoftrafficbetweendifferentdomains
• End-to-enddelayguarantees• FlexibleFHprocessingsplits
– BBUProcessingconstraints– RUDemandConstraints
• Transportnetworkcapacityrequirements– Constraintsimposedbytheelastic
opticalnetwork
LargeDCs Internet
Metro
Core
Metro
MiniDCs
Mid-DCs
Micro-DCs
WirelessAccessMacro-cells
ControllerResources
NFVI
VIM
ControllerResources
NFVI
ControllerResources
NFVI
ControllerResources
NFVI
VIM
VIM
VIM
Frontlhaul
VNF
VNF
SSD
RAMGPP SPPPNF
PNF
VNF SSD
RAMGPP SPP
PNF
PNF
VNF
VNF
VNFBackhaul
vBBU vBBU
Frontlhaul
Optical Networking: RU-BBU interconnection
EdgeNode
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUXBV-WSS
BV-WSS
BV-WSS
FastSwitch
moduleλ1
FastSwitch
moduleλ2
FastSwitch
moduleλΝ
MUX
MUX
c)
CoreNode
RU1
RU3CoreNode
CoreNode
EdgeNode
SPPSPP
GPPGPPToR
EdgeNode
SPPSPPSPP
SPP
ToR
SPP
GPPGPP
SPP
ToR ToR
RU2
OpticalNetwork DataCentre
Tx
ETHTx
I/QTx
ETHTx
I/QTx
ETHTx
I/QTx
ETHTx
I/QTx
ETHTx
I/QTx
BVT
BVT
λ1
λ2
λΝ
λ1
λ2
λΝ
ETHTx
I/QTx
Switch
SYNC
BV-WSS
Multi-protocolInterfa
ce
GPPSPP SPP SPP
Switch Switch
d)
SPPSPP
GPPGPP
SPPSPP
GPPGPP
ToREdgeNode
BVT
λ1
λ2
λΝ
SwitchSYNC
Multi-protocolInterface
a)
Key Network Component
• Interfacingbetweendifferenttechnologydomains:– Scheduling– Aggregation/De-Aggregation– Trafficadaptation– QoS Mapping
Rx FIFO 1
Rx FIFO 2Buffer
Tx FIFO λ1
Tx FIFO λ2
Tx FIFO λ3
Tx FIFO λ4
TSON
FIFO #1
FIFO #1
….
FIFO #M
DC
Fixed Traffic
Mob
ile U
sers
Rx FIFO 3
Rx FIFO 4Scheduler
Buffer
Aggregator
FIFO #1
FIFO #1
….
FIFO #N
Wireless
Buffer
Rx FIFO 1
Rx FIFO 2
Rx FIFO 3
Rx FIFO 4De
Aggregation
Buffer