milcom10 t10 optical ethernet sharma davari 2010 11 01

11

Understanding Intelligent Military-Grade Optical Ethernet Networks:

A Versatile Solution for Achieving DoD’s Net-Centric Operations Strategy

Vishal Sharma, Ph.D.Principal Technologist &

ConsultantMetanoia, Inc.

[email protected] (p)/650-641-0086 (f)

Shahram Davari, MASc.Associate Technical Director,

Network SwitchingBroadcom, Inc.

[email protected] (p)

mailto:[email protected]

mailto:[email protected]

Understanding Understanding Intelligent Military-Intelligent Military-Grade Optical Ethernet NetworksGrade Optical Ethernet Networks::

A Versatile Solution for Achieving DoD’s Net-A Versatile Solution for Achieving DoD’s Net-Centric Operations StrategyCentric Operations Strategy

© Copyright 2010All Rights Reserved

Shahram Davari, MASc.Associate Technical Director, Network SwitchingBroadcom, [email protected] (p)

Metanoia, Inc.Critical Systems Thinking™

Vishal Sharma, Ph.D.Principal Technologist & ConsultantMetanoia, Inc. [email protected] (p)/650-641-0086 (f)

Milcom’10, October 31-Nov 3, 2010, San Jose, CA 3Copyright 2010All Rights Reserved

What We Will Discuss in This Tutorial

Elements of DoD’s Net-Centric Data Strategy – key attributes and goals

Requirements and Attributes of Military-Grade Networks

Implications of the Above for Underlying Technology System Architecture and Features Network Architecture and Design

Why Discuss Ethernet? Its Benefits and Applications

Optical Ethernet

3 Roles of Ethernet – Service, Transport, and PHY Carrier Ethernet and Optical Ethernet

Macro-Architectural Options for Building MAN/WAN Interconnects & Key Operational Principles

Key Developments Valuable for Military Adoption of Optical Ethernet

How Optical Ethernet Technology meets the Initial Requirements

Attributes and Goals of DoD’s Attributes and Goals of DoD’s Net-Centric Data StrategyNet-Centric Data Strategy



Core Elements of DoDs Net-Centric Operations/Data Strategy (NCDS)

Proactively Collect User-Feedback for Improvements

Key Attributes of DoDs Net-Centric Data

Strategy

Handle Info. only Once for Efficiency Visibility to a Wide Audience

Rich, Descriptive Meta-data for Understandability

Shared-spaces for Posting and Efficient Access

Post-and-Process in Parallel

Rapid & Precise Discovery of Data

Facilitate Repurposing – Separate Data from Applications


Strategic Goals of DoD’s NCDS

- Establish procedures & policies for effective data sharing

- Embed data-sharing precepts in the organization

Communities of Interest

Strategic Goals of the Net-

Centric Data Strategy

InstitutionalizedVisible

Understandable

Trusted

Interoperable

- Data is trustworthy

- Data integrity & quality is assured by backing of a reliable organization/authority

- Make meaning & purpose of data clear via use of meta-data

- Discoverable

- Facilitate interaction with data for analysis and decision-making insight

- De-centralize data management to dynamically formed user groups

- Allow prioritization/collaboration on data, based on immediate operational needs

- Furnish infrastructure for self-synchronization

- Shareability of data, while preserving accuracy, integrity, usability

- Understandability via semantic and structural meta-data

Responsive

- React to fulfill user needs

- Satisfy needs relative to performance, content coverage & quality

Accessible

- Ease of reaching data location

- # of users who can consume data

Military-Grade Networks: Military-Grade Networks: Requirements & AttributesRequirements & Attributes



Key Requirements of Military-Grade Networks

- Accommodate multiple access technologies/media – copper, fiber, coax, TDM, satellite, wireless

- Uniformly aggregate traffic onto the metro/core network

Simultaneous Support of Legacy & Advanced Services

Military-Grade Networks:

Requirements

Rugged

ReliableManageable

Highly Available

- Controllable delay, jitter, and loss

- Flexible bandwidth distribution across diverse users/applications

- Resilient to failures

- Ability to recover automatically in min. time

- Hardened for harsh environments – extreme weather, demanding conditions

- Need to operate in constrained spaces

- OAM capability

- Ability to control network elements & resources

Diverse Last-Mile Access

- Uptime: 99.9999% or more

- Fast error detection and recovery

Secure

- Reliable, uncorrupted data

- Tamper-resistant, high-integrity data

Hard QoS & Determinism

- Support legacy voice, POTS, low-speed satellite backhaul links

- In parallel, allow for rich, multi-media traffic, video commn, sensor data

Fast Connection Setup

- For dynamic and quick call setup

- Setting up commun. over underlying infrastructure

Implications for Technology, and Implications for Technology, and System & Network ArchitecturesSystem & Network Architectures



Implications of NCDS Requirements (1)

Implications for: Technology System Design Network Architecture Property

1 Scalability (# locations, # users)

- Large address space to support many end-nodes - Capability to create hierarchy - Control Plane for discovery & topology learning

- Large memory/processing for address & routing tables - Capacity for large # of tunnels

- Accommodate many end-nodes - Hierarchical design & traffic engineering - Support wide geographic reach, seamless across access, metro, core

2 Security (data integrity, trust)

- E2e, segment, and/or Link layer (local) security - Isolate different users or user classes - Enable detection of breaches

- Support encryption, authentication, ACLs - DPI on line cards - User data isolation - Intelligent memory partitioning across users/functions - Provision against DoS/security attacks

- Admission control - Authentication - Architecture that integrates firewalls, appliances with DPI

3 Manageability (of network and data)

- Provide robust OAM tools - Management interface & protocols (e.g. ELMI)

- Support OAM tools/mechanisms - Permit remote access & mgt.

- Allow for Out-of-band (OOB) control - Support a data communication network (DCN)


Implications of NCDS Requirements (2)


4

Dynamic setup and control of communications (within & across COIs)

- Signaling - Dynamic/static tunnel setup - NMS configuration features

- Discovery - Signaling, CP features - Dynamic joining of mcast groups (e.g. IGMP)

- OOB network for signaling (if needed)

5

Native mp2mp, p2mp communication (for many-to-many xchanges, multicast)

- Native broadcast, multicast capability - Mcast signaling & QoS

- System-level brdcast, mcast with intelligent replication - Multicast signaling support - mcast group creation/deletion

- Strategic placement of servers (close to consumers) - Redundancy of data (servers) - Support redundant & disjoint network paths

6

High-Speed at low cost (rapid communication)

- Support multiple i/f speeds - Allow link bundling to enable higher speeds - Have standards for evolving speeds - Backward compatibility with earlier i/fs

- Large fabrics - Versatile, dense line cards - High-rate processing - Low power consumption

- High-speed links -- fiber - Support WDM - Enable link aggregation


Implications of Military-Grade Network Requirements (1)


1 Rugged

- Ubiquitous, with wide reach and minimal constraints - Delivarable over robust media, e.g. fiber

- Robust conduction cooling - Intelligent use of CPUs - Off-load complex processing -- security, protocols -- to central entity or add-on

- Built with robust media E.g. fiber -- inert, free from EMI/EFI

2 Secure

- Standards for encryption, security that are widely accepted/realizable, available - Tunnel user data in real/virtual tunnels to effect isolation - Raise alarm/signal when data is tampered with

- Data plane and control plane robust to DDoS - Apply hardware-based encryption - Isolate users via memory partitioning, queue mgt., tunnels to minimize data impact

- Network and overlay mgt. architecture must resist hacking/tampering - Have rapid alarm propagation

3 Reliable

- Stds for signaling -- for restoration - Setup & control multiple paths via signaling/NMS - Detect/react to faults, mis-routed data - OAM capabilities such as: connectivity check, loopback, link trace

- Hardware/software redundancy - e.g. LCs, fabrics, power supplies - Software redundancy - NSF, NSR, hitless upgrades - Ability to detect/react to failures

- Support topologies supporting redundancy in data routing - Dual-homing, link aggregation (e.g. LAG), multipath (e.g. ECMP) support - 1+1, 1:1, 1:N, ring, mesh protection



Implications for: Technology System Design Network Architecture

Property

4 Hard QoS + Determinism

- Support virtualization of network b/w (e.g. via tunnels, VLANs) - Ability (in technology, e.g. pkt hdrs) to mark, seggregate, prioritize, aggregate traffic - Support perf. measurement OAM

- Traffic isolation via queues, scheduling - Separate tables/memories to segregate traffic of different priorities, classes, apps. - Signal tunnels, and control/ manage tunnels

- Support provisioning and dimensioning - CAC to regulate traffic vols. - Traffic engineering to support traffic placement

5 Manageable

- Management constructs for config, monitoring - Measure loss, delay - Have loopback, link trace, continuity check (e.g. Y1731)

- Control access to/sharing of system resources between different user types - Create/config policy - Gather stats, diagnose problems

- Support remote config. & monitoring - OOB or in-band DCN - Hierarchical design

6 Available

- Error detection & config of multiple alarms - Multipath routing/switching - Rapid switchover on failure

- Fast error detection at L1/L2/L3 - Detect h/w, s/w errors - L1/L2/L3 integration for fault alarming - Hardware/software features to aid redundancy

- Support alternate routes/paths - Architecture to enable rapid recovery from failures (meshy-architecture) - Support intelligent/flexible multi-layer protection




7 Diverse Last-Mile Access

- High-speed, cheap, easily upgradable - Simple management or unmanaged - Support aggregation of traffic, while keeping different traffic types/classes seggregated

- Multi-service capable to support variety of interfaces -- TDM, ATM, FR, IP, EPON/GPON -- and protocols - Support vast range of data rates - Ability to aggregate traffic (Appropriate processing in h/w and s/w Ability to queue & route data appropriately)

- Intelligent interworking (type, #, placement of devices) - Provide for aggregation points/on-ramps for termination of diverse traffic and transfer to a common (Ethernet, IP/MPLS) core - Enable clock distribution

8 Support Legacy & Advanced Services

- Advanced security mechanisms - Smart OAM - Virtual partitioning of network resources (for communities) - Scalable multicasting - Sophisticated security mechs.

- Support VPNs to facilitate COIs - Support ckt emulation, clock distribution - Advanced protection/restoration - Ablility to introduce new services by minimal system upgrades (e.g. just add/modify one LC)

- Architect for incremental introduction of advanced services

9 Low Cost

- Uses technologies with mass adoption in non-military setting (e.g. Ethernet, IP, MPLS) - Benefit from operational experience, cost reductions - Use technologies with accumulated deployment experience

- Leverage COTS - Use std. building blocks/sub-systems, components to benefit from adoption of vol. components (or hardened variants)

- Versatile arch. -- uses technologies optimized per segment - Intelligent policy enforcement (via policy servers) - Plug and play operation - Powerful signaling and control

Why Discuss Ethernet?Why Discuss Ethernet?It’s Benefits and ApplicationsIt’s Benefits and Applications



Why Ethernet? Some Key Benefits … Mature technology

3 decades of operational experience, ~300M+ ports sold in 2008 alone!

Low-cost Mass usage lowers cost, so

compelling to use wherever possible

High-bit rates & Range of speeds 10 Mbps to 10 Gbps! (40-100 Gbps

underway) – 3 orders of magnitude

Versatile Usable as service, transport, PHY More discussion of this ahead ...

Provides consistent technology from edge-to-core Extends reach from LANMAN WAN

Solves both networking & real-time interconnect needs in military environ.

Native support of IP Imp. for GIG and net-centric warfare Simple IP address management

Self-replacement capability Largely backward compatible Easy upgrades, integration of legacy

systems

Widely available COTS ecosystem Easy to adapt commercial h/w & s/w

for military use

Ethernet expertise widely available Network design, planning, architecture Network engineering, troubleshooting

Practically unlimited interoperability


Representative Applications of Ethernet in the Military Switched Ethernet operates as:

Networking infrastructure for MAN/WAN Real-time fabric interconnect in military systems, warfare systems, & military installments Critical building block for military devices

1-10 Gb/s Ethernet used as “fat-pipe” between sub-systems

Intelligent Ethernet transport adopted for: Support of IP-centric service requirements Evolution of wireless & fixed-line infrastructures Explicitly defined native Ethernet connections w/ reserved resources, dedicated protection

Multi-layer Ethernet switches employed in support of DoD plans to leverage IPv6

Ethernet technology facilitates delivery of: Real-time imaging, sensor data, video Secure mission-critical defense communication

Utilized for furnishing precision timing & sub-microsecond synchronization

USS Ronald Reagan

AH-64 ApacheAH-64 Apache

Optical Ethernet Explained: Optical Ethernet Explained: Three Roles and Its Three Roles and Its

CharacteristicsCharacteristics



Versatile Packet Networking with Ethernet

Ethernet technology can play one of three roles in a data network

NetworkComponent

MEF

Standards Organization

Involved

Transport

Service

PHY

Technology/Standard

Carrier Ethernet

IETF

IEEE

ITU-T

MPLS-TP

PBB, PBB-TE

OTN-transport part

IEEE

ITU-T

1GE/10GE/100GE

OTN-PHY part

Ethernet Service – offered to end-customer, runs e2e, where traffic flow into/out of customer systems comprises Ethernet frames

Ethernet Transport - Ability to switch/route Ethernet frames of an Ethernet service, b/ween network nodes by setting up connection-oriented, traffic engineered paths in the network with deterministic perf.

Ethernet PHY – framing and timing of actual bits of the Ethernet frame, and their TX over the physical medium to connect switches at the physical layer


A Word on Connection-Oriented Ethernet (COE)

Ethernet transport enables the realization of COE

COE – set of control-plane protocols & data-plane settings that create a connection-oriented capability to transfer Eth frames

Ethernet transport could involve:

L2 transport -- Switching/routing traffic (data frames) by

Enhancing Ethernet technology – e.g. PBB-TE (802.1aq)

Using a different technology – e.g. MPLS, MPLS-TP

L1 transport – switching/routing traffic at the physical layer (e.g. optical channel data unit (ODU) level) by

Embedding in a transport networking layer, such as ITU-T’s G.709 OTN


Optical Ethernet Network Defined Network spanning a MAN/WAN that offers a carrier-grade Ethernet service,

running on a COE transport infrastructure over an optical PHY Optical PHY: OTN’s optical channel or an Ethernet PHY over optics Can be muxed onto fiber using CWDM/DWDM

L2 Transport

Service

L0 PHY

Carrier Ethernet (E-line, E-LAN, E-Tree)

Packet Transport (PBB-TE, MPLS-TP)

SONET/SDH, OTN transport

OTN-PHY partIEEE-Ethernet PHY)

L1 Transport

“Optical Ethernet” Layers

Technology Examples

For p2p services

For p2p or mp2mp services

Relationship of the Layers andtheir corresponding entities


OK, So What is Carrier Ethernet?

Carrier Ethernet is therefore the service component of optical Ethernet networks

Courtesy: Metro Ethernet Forum


How Optical Ethernet Relates to Carrier Ethernet

Carrier Ethernet: defined by MEF in 2004-05 as “Ubiquitous carrier-grade Ethernet service with 5 attributes”: Standardized Services (better thought of as building blocks)

Uniformly defined core services, building blocks for applications

E-line, E-LAN, E-Tree (illustrated ahead)

Scalability Span local, access, national, global range, with millions of MACs & UNIs

Reliability Detect & recover from errors/faults, without impacting customers

Hard QoS E2e performance for loss, delay, jitter, and b/w matching requirements of

voice, video, data traffic over heterogeneous networks

Service Management Robust, standards-based, vendor-independent OAM to monitor, diagnose,

manage networks offering Carrier Ethernet service


MEF’s Service Definitions or Building Blocks

MEF building blocks defined in terms of Ethernet Virtual Connections (EVCs)

EVC

Association of two or more User Network Interfaces (UNIs) at the edge of metro Ethernet network (MEN) cloud

Exchange of Ethernet frames limited to the UNI’s in the EVC

Three building blocks specified

E-Line – p2p EVC

E-LAN – mp2mp EVC

E-Tree – p2mp EVC


MEF’s Building Blocks Illustrated

26

EVC1

EVC2

Root

Leaf

Leaf

Point-to-Point EVC (E-Line) Multipoint-to-Multipoint EVC (E-LAN)

Rooted-Multipoint EVC (E-Tree)

EVC1

EVC2

Root

Leaf

Leaf

EVC1

EVC2

Root

Leaf

Leaf

EVC1

EVC2

Root

Leaf

Leaf


Putting it Together: Optical Ethernet Network Components in Operation

Service

Transport

PHY

E-LAN Service

Ethernet Service (end-to-end; what the

user perceives)

Ethernet Transport (what the cloud delivers; the

“pipe” and its routing)

PHY (how the bits are transported

between systems)

Framing, timing, and optical muxing

Ethernet Service

PHY Layer(physical link, fiber)

Switching/Routing

Optical (WDM) transport

Macro-Architectural Options for Macro-Architectural Options for Building MAN/WAN Inter-Building MAN/WAN Inter-

connects & Design Principles connects & Design Principles InvolvedInvolved



A Word on Network Architecture

Ultimate goal of a network: to provide end-to-end connectivity between two entities

E.g. client-server, user-to-user, …

Path between entities has many segments, comprising

Access, aggregation, metro/edge, core

Different technologies can be used in each segment, depending on that segment’s requirements


Applicability of Ethernet to Network Segments

Access

Cost Very cheap

High-speed, vast range (10 Mbps – 1 Gbps)Speed

ManageabilityLittle or no mgt. needed

(plug-and-play)Supports ELMI

Relatively cheap

Aggregation Core

Sophisticated systems increase cost

High speeds/feeds, 1 Gb/s – 10 Gb/s, link agg.

High speeds, 1 Gb/s – 100 Gb/s, LAG

Comprehensive OAM portfolio

Fault & Performance Mgt. OAM

LAG and Dual Homing (IEEE Work-in-Progress)

Via RSTP, MSTP, ring protection (G.8032)

Linear protection (G.8031), Traffic

engineering

Supports 4K services/access link

Allows hierarchy (MAC-in-MAC), Upto 16M

services

Via hierarchy, with inter-operability with IP/MPLS (PBB-VPLS interworking)

Redundancy

Scalability

Works over diverse access media (E.g. fiber,

Cu, wireless, coax, ...)

Multiple logical rings, mesh natively supported,

native multicast

Supports TE, routing extensions (e.g. PLSB)Notable Features

Network Segment

Parameters


Flexibility with Ethernet

Ethernet has features that make it suitable for the 3 key segments – depending on the operator’s need

Adaptability of Ethernet implies

Ethernet is not always needed end-to-end

Usable in segments where it makes sense

Incrementally extendable to other segments

Interoperability of Ethernet can inter-work with other technologies for optimum realization of services


Network Architecture Options with Optical Ethernet

In the following, we

Discuss key architectural options using Ethernet & optical Ethernet

Show how Ethernet migrates from the access (it’s forte) to the metro and core

Present the merits & assessment of each architecture


Ethernet in Access: Operation & Protocol Stack

X X

N-PE N-PE

IP/MPLS

CECE

Q-in-Q

MPLS/PW MPLS/PWMPLS/PW

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload Payload

LSP-Label

VC-Label

C-DA

C-SA

S/C-Tag

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload

LSP-Label

VC-Label

LSP-Label

VC-Label

LSP-Label

VC-Label

LSP-Label

VC-Label

Spoke PWs per VPLS instance

IB-BEBMPLSMPLS

LSRU-PE LSR U-PE

AccessAccess

Metro MetroCore


Ethernet in Access: Evaluation

Doable today! and allows gradual “upgrade” to Ethernet in metro and/or core

Cheap, flexible, convenient – uses familiar Ethernet tech. in access

Supports up to 2M services (due to 20b MPLS label) – not scalable

Needs PWs/tunnels e2e, u-PE to u-PE – potentially millions – which could become unmanageable

Metro & core networks can be anything, but are typically IP/MPLS


Ethernet in Access & Metro: Operation & Protocol Stack

B-Tag

X X

N-PE N-PE

IP/MPLSPBB PBB

B-BEBU-PE B-BEB U-PECECE

Ethernet EthernetMPLS/PW

B-DA

B-SA

B-Tag

I-Tag

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload Payload

B-DA

B-SA

I-Tag

C-DA

C-SA

S/C-Tag

B-DA

B-SA

I-Tag

C-DA

C-SA

S/C-Tag

Payload

B-DA

B-SA

I-Tag

C-DA

C-SA

S/C-Tag

Payload

B-DA

B-SA

B-Tag

I-Tag

C-DA

C-SA

S/C-Tag

Payload

C-DA

C-SA

S/C-Tag

Payload

Must support B-BEBand VPLS capability

B-VID locally significant in PBB, not sent over core

B-BEB B-BEB

Internal B-VID, enables I-SID

bundlingVD-Label

LSP-LabelB-BEB removes

PBB-specificB-Tag

PBB

AccessAccess

Metro MetroCore

PBB


Ethernet in Access & Metro: Evaluation

Implementable today, with selected hardware/software

Allows gradual “upgrade” to Ethernet in core, if needed

Cheaper, easier, lower cost & maintenance than previous option (Ethernet in access only)

Metro PBB network enables scaling of services, while reducing processing/memory burden on metro/core devices

Core network can be anything, but is typically IP/MPLS


Ethernet Everywhere: Protocol Stack

CE

Access (802.1ad)

Metro/Aggregation (802.1ah)

Core (802.1Qay)

Metro/Aggregation (802.1ah)

Access(802.1ad)

PB

IB-BEB

BCB

BCB

PE

PE

B-BEB

B-BEB

BCB

B-BEB B-BEB

BCB

IB-BEB

PB

PE

PE

B-BEB

802.1ad/Q-in-Qencapsulation

802.1ahencapsulation

802.1ahdecapsulation

802.1ad/Q-in-Qdecapsulation

C-DA

C-SA

C-Tag

Payload

C-SA

S-Tag

C-Tag

Payload

C-DA

C-SA

S-Tag

C-Tag

Payload

C-DA

B-DA

B-SA

B-Tag

I-Tag

C-SA

S-Tag

C-Tag

Payload

C-DA

B-DA

B-SA

B-Tag

I-Tag

C-SA

S-Tag

C-Tag

Payload

C-DA

B-DA

B-SA

B-Tag

I-Tag

C-SA

S-Tag

C-Tag

Payload

C-DA

C-DA

C-SA

C-Tag

Payload

CEIB-BEB

IB-BEB

CE

CE

B-BEB

BCB

B-BEB

Provider Bridging (PBB) Provider Backbone Bridging (PBB)

Provider Backbone Bridging (PBB)

Provider Bridging (PBB)

PBB – Traffic Engineered (PBB-TE)

Switching based on pre-configured fwding tables

- Pinned paths- Based only on B-DA, B-SA, B-Tag - No STP- No MAC learning

Last MileLast Mile


Ethernet Everywhere: Evaluation

Uses proven, uniform technology throughout

Ability to transport Ethernet & IP services (since Ethernet supports IP)

Benefits Easy procurement, management, troubleshooting

Cost efficiencies (opex) from understanding, managing a single technology in the network

No technology interworking required!

Supports link, segment, and e2e (service) OAM with one OAM technology


Ethernet in Mobile Backhaul

Mobile backhaul architectures derive from the previous basic types

We examine them separately due to their unique needs:

Interface with the core network

Timing and synchronization requirements

Evolution requirements – from TDM or ATM to IP/MPLS and/or Ethernet


Evolution of Cellular Technology and Backhaul Types

Network Speed Interface

GSM/GPRS

EDGE

3G (UMTS/WCDMA) R3, R4

3G, R5 (HSDPA), R6 (HSUPA)

LTE R8 (20 Mhz)

CDMA1X-RTT

CDMA EV-DORev A/B

WiMAX (10 Mhz)

56-114 Kbps TDM

236 – 473 Kbps

384 Kbps Uplink14.4 Kbps Downlink

500 Mbps Uplink>100 Mbps Downlink

100 Kbps

1.8 Mbps Uplink1.8 to 5 Mbps Downlink

384 Kbps Uplink384 Kbps Downlink

TDM

ATM

IP/Ethernet

IP/Ethernet

IP/Ethernet

TDM

IP/Ethernet50 Mbps

Backhaul Types

2G

2.5G

3G

4G

Legend


Mobile Backhaul Components Backhaul network – defined as the network that connects

Base Transceiver Station (BTS, or Base Station) to Base Station Controller (BSC) in 3GPP2 – GSM-based cellular networks

Node-B to Radio Network Controller (RNC) in 3GPP – CDMA-based cellular networks

Traditional backhaul networks have used ... E1/T1 leased lines SONET/SDH TDM channels (for higher rate aggregation)

Mobile transport infrastructure has hitherto been ... Microwave links Optical fiber with SDH/SONET

Evolution to packet-based wireless services creates a push for the transport itself to be packet-based: Ethernet or IP/MPLS or a combination


Traditional Backhaul Evolution

SDH/SONET Network

BSC

RNC

TDMTI/EI Cellsite

Gateway

ATM

nxE1

T1/E1/STM

E1

ATM

3G BTS

2G BTS

ATM Switch

SONET/SDH XConnect

SONET/SDH XConnect

Separate transmission facilities for different technologies (TDM and packets)


Evolved Backhaul Network

Carrier EthernetNetwork

BSC

TDMTI/EI Cellsite

Gateway

ATM

nxE110/100/1GE

Ethernet

1/10GE Ethernet

3G BTS

2G BTS

IP/Ethernet Switch/Router

IP/Ethernet Switch/Router

Ethernet

3G/4G BTS

PE PECE

To Wireless Core

Common transmission infrastructure for different technologies (TDM and packets)


A Quick Primer on PseudoWires

PE1 PE2

VSI VSI

ACs AC1

ACn

Tunnel LSP

1

VPN_ID = AVPN_ID = A

2

3 Discovery

Targeted LDP4

VPN_ID = ALabel = 2004

5

Label Mapping VPN_ID = A

Label = 10045

Label Mapping

VC_LSP (2004)

VC_LSP (1004)

PW established

6

4. Targeted LDP session established

5. Mapping of label for the VC LSP (unidirectional virtual circuit (VC)) exchanged between end nodes

6. PW established, data transfer enabled

1. Bind attachment circuit to Virtual Switching Interface inside PE router

2. Assign each PE node a VPN id.

3. Nodes discover each other


Pseudowires (PW) for Legacy Transport

Carrier EthernetNetwork

BSC

TDMTI/EI

Cellsite Gateway

ATM

nxE1

10/100/1GE Ethernet

1/10GE Ethernet

3G BTS

2G BTS

Ethernet

3G/4G BTS

PE PECE

To Wireless CoreAC

AC

PSN Tunnel

PW

AC: Attachment CktPE: Provider Edge

CE : Customer Edge (BTS)BSC: Base Station Controller

Encapsulation

Structure-Agnostic TDM-over-IP (SAToIP) (RFC 4553)

Structure-Aware TDM Circuit Emulation (CESoPSN) (RFC 5086)

ATMoPSN (RFC 4717)

PSN Tunnels May be IP/MPLS, T-MPLS/MPLS-TP, or

PB/PBB/PBB-TE based

PW Signaling

IEEE 1588-based timing distribution supported

SyncE (Synchronous Ethernet)


MEF Services for Mobile Backhaul

Metro EthernetEVC EVC

RNC

BSC

Service Multiplexing

BTS BTS

RNC

BSC

BTSBTS

BTS

mp2mp EVC

Metro Ethernet

Services muxed at RNC UNI Needed when inter-BS communication is permitted like in LTE/802.16m (WiMAX)

EVPL Service for Backhaul using Metro Ethernet Networks

EVP-LAN Service for Backhaul using Metro Ethernet Networks


MEF Services for Mobile Backhaul

Metro Ethernet

EVC EVC

RNC

BSC

Service Multiplexing

BS/BTS

BS/BTS

BS/BTS

EVP-Tree Service for Backhaul using Metro Ethernet Networks

Key Developments Valuable for Key Developments Valuable for Military Adoption of Optical Military Adoption of Optical

EthernetEthernet



Optical Ethernet: Recent Developments

Ethernet technology evolving rapidly in the last 3-4 years

Multiple standards bodies have created valuable stds in: OAM

Interoperability

Scalability

Reliability

Security

New Services

Last-mile high speed access

Interworking

New capabilities in Ethernet – synchronization/timing, automatic SLA negotiation, Ethernet demarcation devices, Ethernet as xport

We summarize these next, and focus on key ones valuable for the military


Recent Advances in Optical Ethernet Standards: Snapshot

Area Standard and/or Activity Stds. Organization(s)

Reliability/Protection

Linear (G.8031) & ring (G.8032) protection ITU-T SG15

OAMConnectivity Fault Mgt. (802.1ag), Perf. Mgt. (Y. 1731)

IEEE, ITU-T SG 15

Security LinkSec, MACSec, Authentication IEEE

ScalabilityHierarchy via Shortest Path Bridging (PLSB)Provider Back-bone Bridging (802.1ah)

IEEE

InterworkingFCoE, Ethernet PWs, Circuit Emulation over

Ethernet (MEF 8)IETF, MEF

New Services E-Tree (p2mp communication for multicast) MEF

Higher-SpeedsFast last mile access (EPON, 802.11n), HS i/fs (40G,100G)

IEEE

New Capabilities

SyncE (link-layer clock distribution)1588v3 (network level time & clock distribution)Demarcation devices (MEF NID)Automatic SLA negotiation (MEF E-LMI)Ethernet as transport (PBB-TE)MPLS-TP (Transport Profile): applicable for COE

IEEE, MEF, IETF


Ethernet Security:LinkSec (MACSec, KeySec) Layer 2 link security standard defined by

MACSec (IEEE 802.1ae) KeySec (IEEE 802.1af)

MACSec provides: Origin authentication Data integrity checking Data confidentiality between two e2e Ethernet switches

MACSec defines a frame format that includes data encapsulation, encryption, authentication

KeySec defines key mgt. protocol for MACSec


MAC Sec Packet Format

TCI = Tag Control Info.AN=Association No.SL = Short Length (i.e. no SCI inserted)PN = Packet No.SCI= Secure Channel ID (optional)


Ethernet OAM

Ethernet OAM supports Layer (domain) Monitoring Up to 8 layer levels (domains) per VLAN

Ethernet OAM has tools for: Fault Management (802.1ag): CCM, LB, LT, AIS, RDI

CCM: Continuity Check Message – verifies one-way connectivity LB: Loop Back – checks 2-way (round trip) connectivity LT: Link Trace – provides path (nodes) between nodes A & B AIS: Alarm Indication Signal – provides fwd alarm propagation RDI: Reverse Defect Indication – provides rev alarm propagation

Performance Measurement (Y.1731): LM, DM LM: Loss Measurement – measures loss on an EVC DM: Delay Measurement – measures latency on an EV


Ethernet OAM & Maintenance Domains

Access Core Access

Customer CustomerService Provider

Customer OAM Domain

Provider OAM Domain

Operator OAM Domain Operator OAM Domain

Operator OAM Domain

Independent OAM can be run in each OAM domain for the same VLAN

IEEE provides for 8 levels of Maintenance Domains – allows a level to be assigned to each entity – customer, provider, operator


Ethernet OAM: Loopback (LB) Example for Provider & Operator Domains

Access Core Access

Customer CustomerService Provider

Customer OAM Domain

Provider OAM Domain

Operator OAM Domain Operator OAM Domain

Operator OAM Domain

Independent OAM can be run in each OAM domain for the same VLAN

We show operator, provider, and customer loopback examples above

E2e Ethernet path

Provider LB

Operator LBs

Customer LB


Synchronization in IEEE 1588

1588: a protocol designed to synchronize real-time clocks in the nodes of a distributed system that communicate using a network

Synchronizes both – clock & Time-of-Day (SyncE only synchronizes clock)

Network

Master Slave/Boundary Slave/Boundary


IEEE 1588 Synchronization Operation & Clock Offset Computation

Clock Offset Computation

MS delay = t2 – t1SM delay = t4 – t3

offset = {MS_delay –SM_delay}/2

1588 Operation

How Optical Ethernet Meets Key How Optical Ethernet Meets Key Technology Requirements of Technology Requirements of

Military NetworksMilitary Networks



Role of Ethernet Technology

Ethernet component provides several key capabilities

Native mp2mp communication

Easily creates communities of interest (COIs)

Segregation of traffic and users

Via VLANs (802.1ad) or B-VID/B-VLAN tags (802.1aq)

Enables use a common infrastructure for multiple virtual networks


Role of Optical Technology

Optical component complements Ethernet packet technology, providing strengths where Ethernet does not suffice

Robustness against interference/EMI

Tremendous bandwidth scalability Using an optical fiber transmission medium Via WDM technology, without needed additional fiber

Connection-oriented nature Allows for traffic engineering Sophisticated, ultra-fast protection/restoration

Long reach/range Reliable communication over long distances

Facilitates deterministic timing/performance


Suitability of Optical Ethernet for the Military (1)

Military Network Requirement

How Today’s Optical Ethernet Technology Meets It

1 Scalability

- Hierarchy – via MAC-in-MAC encapsulation - Routing & Topology capability – PLSB, TRILL (MAC learning in CP)

2 Security

- MACSec – providing e2e security between nodes - ACLs – based on address, VLAN, port, … - Queueing per VLAN, class, app., in systems

3 Manageability

- Extensive OAM for fault & perf. management - Service-level and link-level OAM, with hierarchy - OOB management capability - ELMI negotiation at UNI

4 Dynamic Setup & Control

- RSTP variants - MSTP - ELMI for negotiation at UNI - LACP helps setup link aggregation groups - IS-IS in control plane for network topology control

5 Mp2mp and p2mp communication

- Inherently mp2mp technology - E-Tree service from MEF

6 Low-Cost - Economical deployment - Capex $1/ 1 Mb/s, which is ~1/4th of TDM circuit cost


Suitability of Optical Ethernet for the Military (2)

Military Network Requirement

How Today’s Optical Ethernet Technology Meets It

7 Reliability - ITU-T link and ring protection - EAPS (Ethernet Automatic Protection Switching), RFC 3619 - Link Aggregation (LAG)

8 QoS

- VLANs for virtualization - Use of “p” bits for prioritization - Bandwidth profiles (MEF) for queueing - Per VLAN, per class traffic management - Policing, shaping, dropping, metering, marking within

systems for differentiation between traffic

9 Availability - Linear + Ring protection - EoWDM to increase reach, while decreasing BER - Dual homing in access & E-NNI (network interfaces)

10 Diverse Last-Mile Access

- P2p Ethernet - Wi-Fi access - WiMAX access - EPON

11 Support of Legacy Services

- Circuit Emulation over Ethernet (MEF8, SATOP, CESoPSN)

- Use of EtherType allows native encapsulation (of different traffic types) within Ethernet. E.g. FCoE, PPPoE

12 Clock Distribution - SyncE - IEEE 1588v2

Summary and ConclusionSummary and Conclusion



Wrapping it Up ...

Optical Ethernet is today a well-established & well-known technology, with many capabilities

New capabilities being rapidly added, due to its versatility and popularity

Usable in access, metro, core, in mobile backhaul, data centers, ...

Interoperable – so can be mixed-and-matched with other technologies

Suitable for net-centric, military applications

Adds value in many applications, and a strong candidate to be used where its characteristics fit the application or network segment under consideration

Thank You!Thank You!Questions? Questions?


GlossaryGlossary



Glossary (1)ACL Access Control List

BCB Backbone Core Bridge

BEB Backbone Edge Bridge

B-MAC Backbone MAC

BSC Base Station Controller

BTS Base Transceiver Station

B-VID Backbone Virtual ID

CAC Connection Admission Control

CE Customer Edge

COI Communities of Interest

COTS Common Off-The-Shelf

DA Destination Address

DCN Data Communication Network

DoD Department of Defence

DPI Deep Packet Inspection

DWDM Dense Wavelength Division Multiplexing

e2e End to End

ECMP Equal Cost Multi-Path

ELMI Ethernet Local Management Interface

EPON Ethernet Passive Optical Network

EVC Ethernet Virtual Circuit

GPON Gigabit-capable PON

H-QoS Hierarchical QoS

IEEEInstitution of Electrical and Electronic

Engineers

IETF Internet Engineering Task Force

IGMP Internet Group Management Protocol

I-SID Individual Service ID

LAG Link Aggregation Group

LC Line Card

LDP Label Distribution Protocol

MEF Metro Etherent Forum

MEN Metro Ethernet Network

mp2mp Multi-point to Multi-point

MPLS Multi Protocol Label Switching

MPLS-TPMulti-Protocol Label Switching -

Transport Profile


Glossary (2)MSTP Multiple Spanning Tree Protocol

NGN Next-Generation Network

NMS Network Management System

N-PE Network-facing-Provider Edge device

NSF Non-Stop Forwarding

NSR Non-Stop Routing

OAMOperations, Administration, and Maintenance

ODU Optical Data Unit

OOB Out of Band

OTN Optical Transport Network

p2mp Point to Multi-point

PB Provider Bridging

PBB Provider Backbone Bridging

PBB-TEProvider Backbone Bridging - Traffic Engineering

PE Provider Edge

PHY Physical Layer

PLSB Provide Link State Bridging

PON Passive Optical Network

POTs Plain Old Telephone Service

PSN Packet Switched Network

PW Pseudowire

QoS Quality of Service

RNC Radio Network Controller

RSTP Rapid Spanning Tree Protocol

RSVP-TEResource Reservation Protocol - Traffic Engineering (RSVP protocol with MPLS traffic engineering extensions)

SA Source Address

SDH Synchronous Digital Hierarchy

SONET Synchronous Optical Network

SPT Shortest Path Tree

STP Spanning Tree Protocol

TDM Time Division Multiplexing

TRILLTransparent Interconnection of Lots of Linkshttps://datatracker.ietf.org/wg/trill/charter/


Glossary (3)UNI User Network Interface

U-PE User-facing-Provider Edge device

VLAN Virtual LAN

VPN Virtual Private Network

Appendix: Word on Provider Appendix: Word on Provider Bridging (PB) and Provider Bridging (PB) and Provider Backbone Bridging (PBB)Backbone Bridging (PBB)



Native Ethernet in Metro Access

How does one create the notion of a virtual circuit? VLAN tagging with point-to-point VLAN

VLAN stacking Outer tag service instance; Inner tag individual customer

802.1Q in 802.1Q (Q-in-Q) - IEEE 802.1ad

C-DA: Customer Destination MAC

C-SA: Customer Source MAC

C-TAG: IEEE 802.1q VLAN Tag

C-FCS: Customer FCS

S-TAG: IEEE 802.1ad S-VLAN Tag

C-DA C-TAGC-SA Client data FCSS-TAG

6bytes 6bytes 4bytes 4bytes 4bytes


Customer Network

Customer Network

Customer Network

72

Provider Bridge (IEEE 802.1ad) Architecture

CE: Customer Equipment

UNI: User-to-Network Interface

CES: Core Ethernet Switch/Bridge

P-VLAN: Provider VLAN

UNI-B

CES

CES

CE-A

UNI-A

UNI-C

CE-C

Spanning tree

CE-B

CES


Provider Backbone Bridging (802.1ah)

Encapsulate customer MAC with provider MAC at edge

Edge switch adds 24-bit service tag (I-SID), not VLAN tag

Core switches need only learn edge switch MAC adds.

S-TAG: IEEE 802.1ad S-VLAN Tag

B-DA: IEEE 802.1ah Backbone Destination

B-SA: IEEE 802.1ah Backbone Source MAC

I-TAG: IEEE 802.1ah Service Tag

B-DA B-TAGB-SA I-TAG C-DA C-TAGC-SA Client data B-FCS

6bytes 6bytes 6bytes6bytes4bytes 5bytes 4bytes 4bytes


Provider Backbone Bridging (PBB) Architecture

CPE BCPE ACPE C

Provider backbone network (802.1ah)

CPE BCPE A

802.1ad

CPE BCPE B

802.1q

CPE C

Provider backbone network (802.1ad)

CPE D

CPE DCPE C

CPE A




milcom10 t10 optical ethernet sharma davari 2010 11 01

Technology

ethernet ethernet technology

gbs ethernet

optical ethernet technology

applications optical

ipv6 ethernet technology

data network ethernet

roles of ethernet service

military systems