intelligent optical systems for a 5g world · vehicle a’s view [1] qiu, hang, et al....
TRANSCRIPT
Dan Kilper
June 16, 2020
Intelligent Optical Systems for a 5G World
Overview
• A new network dichotomy
• Optics in Edge Clouds• Disaggregating Optical
Systems• A New Physical Layer for
6G
2
Metro-Edge is the New Long Haul
3
Long Haul shifts to DCI: All about Data Center to Data Center
Massive Enterprise Networks
4
Equinix GXI 2019
Two very different networks
DCI Network
Metro Networks
5
Enterprise Hyperscale DC, IXP
Edge DC, Micro DC, CORD, Access Nodes
In the 5G World
Heterogeneous, pay-as-you-grow, mesh networks
Metro-Regional Edge Cloud Networks
6
Willing to give up capacity for something more
Hyperscale Public Clouds
• Tiered access-metro-long haul network
• All wireless processing at cell site
• Deterministic < 1 ms in building or home
• > 10-20 ms for cloud
7
Telco Mesh Network
Core Cloud
Metro/Core Network
User Eqmt
< 1 ms edge > 20 ms access and core
Bldg
Cell
Access/Backhaul Network
Edge Clouds• Put data center in central office or access-metro boundary
• Cloud RAN: send digitized RF to data center for processing
• Sub-millisecond round trip to edge cloud data center
Edge Cloud Core Cloud
Access Pt
User Eqmt
< 1 ms edge > 1 ms core
BldgCell
8
DCI Network
> 20 ms core
Why Latency Matters: Cloud Assisted Autonomous Vehicles
9
Edge Cloud
Vehicle A’s view
[1] Qiu, Hang, et al. "Augmented Vehicular Reality: Enabling Extended Vision for Future Vehicles." Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications. ACM, 2017.
Vehicle B’s view
Vehicle A’s augmented view
Self-Driving Cars
Vehicle A Vehicle B
E2E Latency ~10 ms required for user QoE
Nokia
Courtesy of D. Raychoudhuri
Other Latency & Bandwidth Sensitive Edge Cloud Applications
10
Line of Sight mmWave Handovers
Phased array antenna synchronization
Coordinated Multi-Point (CoMP) cluster formation
Optics moving data in and between edge data centers
11
64x10G MIMO
100x10G Small Cells
200G Large Cells
Local Wireless Provider
Telco 1
Telco 2
Service Provider A
Service Provider A
Service Provider B
10 λ10 λ
12 λ
Replace some ASIC processing with Photonic Switches
Si MEMS Crossbar, M. Wu UC Berkeley
Network Domains: On-Demand Autonomous Systems
64x10G MIMO
100x10G Small Cells
200G Large Cells
Service Provider A
Service Provider A10 λ
10 λ
12 λ
Service Provider B
Telco 2
Local Wireless Provider
Telco 1
12
New Research Frontier for Optical Systems
• Achieving ultra-low latency• Achieving ubiquitous high capacity
• Efficiently adapting capacity to meet demand
• Using optics in cross-domain, multi-tenant environments
• Meet cost & scalability requirements at the edge
13
Technology is Changing in Response….
14
PON vs DWDM ROADMs
15
T. Pfeiffer, Nokia, ONDM 2018
Problems: low capacity, tree topology, protocol delays, not scalable
Advantages: switched, cheap, relatively simple control
Problems: too expensive, too slow/static, amplifiers, complex control, proprietary single vendor
Advantages: mesh networks, high capacity, scalable in size and capacity
Converging!Simplify control, integrated photonics…Add WDM, mesh…
OpticalAmplifiers:
EDFARaman
MUX DEMUX
OpticalFibers:
SSMF, LEAF
Wavelength Selective Switch:
WSS
<200 km
96 ‘Wavelengths’ (50 GHz)
Reconfigurable Optical Add Drop Multiplexer: ROADM
How do we achieve low cost and scalable, software
controllable, optically amplified and switched systems?
16
Market Driven Computer Dis-AggregationEnabled Hyperscale DC Architecture
Computer Disaggregation Spectrum
18
Max Disaggregated Max Integrated
AppleBuild Your Own from scratch/ hyperscale
Single Integrator: Configure Your Own
Single Integrator: Choose Your Model
Single Integrator: Accessorize(Display, Keyboard)
Choice of Operating System
Disaggregated ROADM Optical Systems!?
19
Disaggregated Optical SystemsAg
greg
ated
Fully Aggregated: Entire transport network acts as a single managed system
End to End System
API
Transponder TransponderTerminal ILA ROADM Terminal
Fully Disaggregated: Everything is a separate network element
APIAPIAPI API API API
Partially Disaggregated: Transponder is one element, Open Line System (OLS) is other
Open Line System
APIAPI API
Disa
ggre
gate
d
20
Courtesy of Victor Lopez, Telefonica
Optical System Disaggregation Spectrum
21
Status Quo
Build Your Own from scratch/ hyperscale
Single Integrator Configure Your Own
Single Integrator Choose Model/ Choose C&M
Single Integrator Accessorize:Open Line Systems
Choice of Control & Management Platform
Max Disaggregated Max Integrated
openROADM
Build Your Own?
Will build your own optical system at the component level ever be a reality?
If a control plane is available, then yes
It’s all about the control plane
Still require expert user: most likely carrier or hyperscale
Some History• Late 90’s: MCI/Globecom tried to build their own systems from
components• ~2000: Unified control plane attempt to merge control of optical
systems into L3 control: Software Disagg.• GMPLS/MPLS was result
• Mid 00’s: JDSU/Nortel introduce ‘generic’ ROADM building block systems
• Late 00’s: Coherent transceivers change system engineering (no dispersion maps, PMD)
• Early 10’s: Enterprises/DC operators buy their own optical networks
• 2020: 5G is here!
23
The Disaggregation Lifecycle
24
Time
Perf
orm
ance
Sustained Growth
More research, Growth of Eco-System/ Supply Chain
Integrated System Growth Slows
Transition to Disaggregation
Research
Fact or Fiction?
What’s holding us back?
25
Control is the problem…
26
SDN: Control for Disaggregated Systems
27
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
DataCenterSDNos
AmplController
WANSDNos
Network SDN OS
Control Data Translator
PCE
PCE
OPL OS
ROADMOA OA
ROADMOAROADM ROADMROADMOCS
OTN/Eth
Layer 2 Controller
Layer 2 Cntrl
OPL OS
OTN/Eth OTN/Eth
Layer 2 Controller
OPL OS
OTN/Eth
Domain 2 Domain 3Domain 1
OpenFlow, Proprietary, TL1
Proprietary System Control
Flat Network OS Control
28
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
DataCenterSDNos
WANSDNos
Network OS
PCE
ROADMOA OA
ROADMOAROADM ROADMROADMOCS
OTN/Eth OTN/Eth OTN/EthOTN/Eth
Domain 2 Domain 3Domain 1
OCM
Power Levelling
Flat Network OS Control
29
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
DataCenterSDNos
WANSDNos
Network OS
PCE
ROADMOA OA
ROADMOAROADM ROADMROADMOCS
OTN/Eth OTN/Eth OTN/EthOTN/Eth
Domain 2 Domain 3Domain 1
OCM
Power Levelling
Ampl. Control
Provisioning Control Chn Discovery
SDN Optical Physical Layer Control
30
Scheduler
Cross-Domain Interface
Policy Manager
Network State/ Abstraction
Ntwk State/ Abstraction
DataCenterSDNos
AmplController
WANSDNos
Network SDN OS
Control Data Translator
PCE
PCE
OPL OS
ROADMOA OA
ROADMOAROADM ROADMROADMOCS
OTN/Eth
Layer 2 Controller
Layer 2 Cntrl
OPL OS
OTN/Eth OTN/Eth
Layer 2 Controller
OPL OS
OTN/Eth
Domain 2 Domain 3Domain 1
Disaggregated System SDN Research
• P. Castoldi, et. al. “Disaggregated optical network control and orchestration of heterogeneous domains” OECC/PSC 2019
• N. Sambo, et. al. “Enabling locally automated reconfigurations in disaggregated networks” OECC/PSC 2019
• Q. P. Van, et. al. “Container-Based Microservices SDN Control Plane for Open Disaggregated Optical Networks” ICTON 2019
• A. Mayoral, et. al. “Multi-layer service provisioning over resilient Software-Defined partially disaggregated networks” JLT to appear 2019
• A. Sgambelluri, et. al. “Fully Disaggregated ROADM White Box with NETCONF/YANG Control, Telemetry, and Machine Learning-based Monitoring” OFC 2018
• M. Shiraiwa, et. al. “Experimental Demonstration of Disaggregated Emergency Optical System for Quick Disaster Recovery” JLT 2018
• L. Gifre, et. al. “Autonomic Disaggregated Multilayer Networking” JOCN 2018
31
MetroHaul: Sgambelluri, et. al. OFC 2018
Transmission Experiments
• Where software meets physics• Starting to see transmission results on
disaggregated systems• Experiments are hard!
• No re-circulating loops!
32
Open Line System Experiment• 8 different transceiver vendors• Up to 1945 km in lab experiments• Use TIP GNpy performance estimation tool
33M. Filer, et. al. JLT 2018
GN Model Performance Prediction Accuracy
TIP, PoliTo used Microsoft commercial development test lab
Physical Layer Control Algorithms• Power drifts over time and new channels are
provisioned: need periodic power adjustments to stay within margins
• Custom simulation to study control performance
34
[1]
A B
[1,2,3,4] [1,4]
[3]
[2]C
D
[3]
[4]
[ ]
Wait for Chn 1
Ready to Adjust
[i,j,k] = channels adjusting upstream
4
511
12
22
23
3130
29
28
27
2625
24
21
13 15
20
18
19
1614
109
17
87
123
6
Kilper & White OFC 2007
Disaggregated Systems Enable Cross-Domain Transmission
Domain 1 Domain 2
Orchestrator
Local Controller
Local Controller
Data
Pla
neCo
ntro
l Pla
neAp
plic
atio
n Pl
ane
ROADMROADM
AgentAgent
User/App
RyuRyu
ONOS
REST
Openflow
CDCP
IXP 1
IXP 2
Control signaling channelOptical data channel
User interface
35
Y. Li, et. al. OFC 2017 Postdeadline; JLT 2018
New component technology with new system control technology
• Integrated photonic PIC OPM to enable continuous per-channel SLA enforcement
• Coordinate SDN QoT estimators to determine path that meets requested performance
TX1x3 Splitter
WSS WSS WSS WSS70km 70km 55km
20km
20km
OPM
RX
OPMWSS
WSS
Agent Agent
ONOS Orchestrator
Domain 1 Domain 2
WSS
OPM
WSS
OPMROADM3
ROADM4 ROADM6
ROADM5ROADM1 ROADM2
IXP
WSS
WSS WSS
WSS
Control Plane
Data Plane
Ryu Controller Ryu Controller
36
OPM: OSNR Monitor
BPF Powermeter∆t
Power meter
Re-Routing Through Controller Negotiation
• Two Scenarios:• Intra-domain re-routing• Inter-domain re-routing
Inter-domain path setup
OSNR monitoring
Intra-domain rerouting
Inter-domain rerouting
Success?
Below threshold?
Success?No
Yes
Yes
Yes
No
Yes
Success?Yes
Traffic fail
Inter-domain traffic request
NoSuccess?
No
Yes
No
No
Inter-domain path computation
Intra-domain impairment?
λ1
Scenario 1:
Scenario 2: λ1
IXP
IXP 37
OSNR monitoring for working path in Domain 1 (good)
OSNR monitoring in Domain 2 (not good)
Intra-domain rerouting in Domain 2
OSNR monitoring for rerouting path in Domain 1 (good)
Scenario 1
OSNR monitoring for working path in
Domain 1 (not good)
Inter-domain rerouting in Domain 1
Intra-domain rerouting in Domain 1
fail
Inter-domain rerouting in Domain 2
OSNR monitoring for rerouting path in Domain 2 (good)
OSNR monitoring for rerouting path in Domain 2 (good)
Scenario 2
6G
• Edge cloud research frontier• Rethink physical layer networks
• What should the new Internet Physical Layer look like?
• Optics & Wireless truly converged• Cross-technology, multi-tenant, and
multi-domain• Embedded electronics with ‘embedded
photonics’• Intelligence and advanced functionality
38
Densification of Wireless Access
• Tiered, North-South architecture & capacity
• Relatively small number of base stations Metro Core
Distribution Rings
PON
P2P
Microwave BH
Access Link
WDMmm Wave
Core OCS
OLT
Long Haul
WDM-PON
Access OCS
Macro RH
Micro/pico RH
BBU/DC
Today
39
Siloed Aggregation
• Patchwork of single operator domains siloed into trees
• Need to go back to core for CoMP on separate trees
Metro Core
Long Haul
Access Link
WDMmm Wave
Core OCS
OLT
WDM-PON
Access OCS
Macro RH
Micro/pico RH
BBU/DC
CORD/BBU pool
Future 1
40
Mesh
• Dense mesh of fiber and fixed wireless connections at the edge
• Optically amplified switched edge fiber systems
• Multi-tenant, multi-domain at physical layer
Metro Core
Long Haul
Access Link
WDMmm Wave
Core OCS
OLT
WDM-PON
Access OCS
Macro RH
Micro/pico RH
BBU/DC
CORD/BBU pool
Future 2
41
Why Mesh?
• Latency improves by staying in physical layer• mesh = more physical layer paths
• East-West capacity for edge applications• CoMP radio, locality aware networks,
rapid mmWave handovers• Better resilience, disaster response• Easier to plan—network chooses the routes,
not the installer
• Less traffic in metro core
42
NYC Fiber
crowncastle.com
Low cost optical switches: PIC-Based ROADMs
• DuPONT PLC ROADM• Photonics Online 2005: “DuPont Photonics
Announces Most Cost-Effective ROADM Solutions Based On Planar Lightwave Circuits”
• JDSU PLC ROADM
44
Amplified optical switching has never left the ground
• Only PONs use fast (< 1 sec) switching in commercial systems
• 25 Years of research and no working solutions in the field!
• Complex channel interactions• Solutions too expensive• Lab prototypes don’t scale to field• Not just about devices: software
problem as well
45
1
10
100
1000
10000
100000
1000000
-4 -3 -2 -1 0 1 2 3 4 5 6 7
Commercial Optical System Switching
• Commercial optical packet switches
• < 10 nodes• Restricted to ring topo.
• Critical gap for metro-edge cloud networks
• Fast circuit switching• Metro Edge: 100-1000’s
nodes over 100’s km• Robust, stable, low cost
46
Switching Frequency, 1/s
Nod
es x
Dia
met
er (1
00km
)
ROADM
Optical ‘Packet’ SwitchingPONs
Amplified Optical Switching
10-4 10-2 1 100 104 106
Switch speed: Min. Sec. mSec µSec
It’s not about devices, its about how the entire system works in
the larger technology eco-system
47
• Harlem city-scale smart city testbed enabling experimentation in optical, wireless and edge cloud computing networks.
• Application domains include AR, VR, connected car, smart city (with high-bandwidth sensing), industrial IoT, …
• Experimentation platform for ultra high BW and low latency tightly coupled with edge computing
Augmented Reality
Smart City + Connected CarCloud
InfrastructureRoadsideAP
Roadway sensors & lighting
In-car guidance display
Image/Video
Industrial Control
48
COSMOS Smart City Testbed
• GHz mmWave capable SDRs with SDN optical whitebox connectivity
• Edge cloud facilities at large nodes and university data centers
• Smart intersections, gigabit centers, IoT toolkits
49
Mininet Optical Emulator• Emulate multi-layer networks
• Including transmission physics
• Run actual SDN code on emulated networks
• Validate against physical/lab networks and extend to larger scale
50
Servers & IP Routers
ROADM Fiber Network
Team lead R. Lantz
SDN Controllers
R. Lantz, et. al. OFC 2020
Conclusions
• Physical layer networks are transforming• Metro-Regional Edge Cloud & Wireless
Networks are Frontier for Optical Systems Research
• Focus on rich functionality, software control, & integrated photonics
• Need system & network scale research—not a single problem to ‘solve’
• 6G is the wireless-optical physical layer Internet
51
The Team
• Dr. Yao Li, Twitter
• Dr. Weiyang Mo, Juniper
• Jiakai Yu
• Shengxiang Zhu
• Tasha Adams
• Mariya Bhopalwala
• Farida Sari
• Haris Khan, Infinera
• Ian Tillman
• Christian Rios
• Aishik Biswas
• Aamir Quraishy
52
Collaborators
• G. Zussman, K. Bergman, Columbia University
• D. Raychaudhuri, I Seskar, Rutgers University
• M. Ruffini, TCD
• M. Veeraraghavan, R. Foley, R. Williams, UVA
• O. Sylvain, Fordham U.
• S. Foster, Georgetown U.
• N. Peyghambarian, R. Norwood, I. Djordjevic, B. Carter, U. Arizona
• C. Banks, B. Lincoln, Silicon Harlem
• R. Lantz
Thank You
Our Group:wp.optics.arizona.edu/dkilper/
COSMOS:www.cosmos-lab.org
CIAN:cian-erc.uawebhost.arizona.edu/
53
This work was supported by the NSF under grants #CNS-1827923, CNS-1650669, CNS-1650685, PFI-AIR-TT: 1601784, ECCS-1547406, CNS-1737453 and ECC-0812072