EoSEoS

Yaakov (J) Stein Chief ScientistRAD Data Communications

Y(J)S EoS Slide 2

Course OutlineCourse Outline

1) Introduction

2) Background - Ethernet

3) Background – HDLC

4) Background - PPP

5) Background - SONET/SDH

6) VCAT

7) LCAS

8) POS (PPP over SONET/SDH – RFC 1619/2615)

9) LAPS

10) GFP

11) Alternatives

Y(J)S EoS Slide 3

IntroductionIntroduction

Y(J)S EoS Slide 4

MotivationMotivationAssume that you are a traditional operator You have an extensive SONET/SDH network This network has cost you Millions-Billions to build This network is highly reliable Your staff is well trained to maintain it You may have not yet reached Return On Investment It supports the service that brings the most revenue – voice It supports the service with the highest margin – leased lines

But suddenly customers are asking for something new “Ethernet handoff”

And new competitors are willing to supply it!

Y(J)S EoS Slide 5

Option 1: install new infrastructureOption 1: install new infrastructure

You may choose to build a new IP/MPLS based network (BT 21CN approach)

Yes – this means significant investment, but this is definitely the future!

But SONET/SDH has comparative advantages: Reliable optical transport Well known technology and protocols Ubiquitous with present operators Many supported data rates (from 1 Mbps to many Gbps) Low overhead Strong OAM (MPLS isn’t there yet …)

So if you replace the existing network How will you handle the service that brings your main income – voice ? You may lose your existing leased line customers You will need to solve the timing distribution problem

And if you keep your existing network You need to maintain two completely different networks !

This sounds problematic !

Y(J)S EoS Slide 6

Option 2: leased linesOption 2: leased lines

You can try to convince these customers to use leased linesThe customer converts traffic into T1/E1 (e.g. by using frame relay) You can supply this service now The major expense is for the customer (who needs FRAD, CSU/DSU, etc.) Leased lines are profitable

But this only worked before the new competitors appeared

You will probably lose these customers !

IWF

IWF

Ethernet Switch

Ethernet Switch

SONETRING

ADM

ADM

Y(J)S EoS Slide 7

Option 3: ATMOption 3: ATM

You can offer ATM serviceThe customer converts traffic into ATM (AAL5) You can supply this service now ATM is a well-known technology ATM is a reliable and high-quality service ATM maps efficiently onto SONET/SDH You may even be able to perform the conversion at your POP (but Ethernet is notoriously hard to transport over distances)

But ATM has its disadvantages ATM has high overhead – but you can only charge for user BW ATM is an additional network

– you will have to train and pay new staff– maintain another operations center

ATM usually carries IP, not native Ethernet traffic

ATM

ATM

Ethernet Switch

Ethernet Switch

SONETRING

ADM

ADM

Y(J)S EoS Slide 8

Option 4: EoSOption 4: EoS

A new choice is Ethernet over SONET/SDH (EoS)

The customer’s Ethernet traffic is transported directly by SONET/SDH You build on your existing network You transport native Ethernet

– needn’t route at network edges– maintain all Ethernet features

New SONET/SDH features make EoS highly efficient

But EoS and related protocols are new technologies You may need to upgrade existing equipment Market hasn’t yet stabilized on one technology

So you will probably need to take this course !

IWF

IWF

Ethernet Switch

Ethernet Switch

SONETRING

Y(J)S EoS Slide 9

World’s ApartWorld’s ApartSONET/SDH is presently the most prevalent transport infrastructure

Ethernet is by far the most popular user data interface

So we need efficient methods for carrying Ethernet over SONET

But Ethernet comes in bursty “frames” (packets) uses basic rates of 10, 100, 1000 Mbps

While SONET/SDH is constant bit rate is designed for various rates such as 1.6, 2.176, 6.784 Mbps

So the job isn’t easy !

Y(J)S EoS Slide 10

Standards we will encounterStandards we will encounter

IEEE 802.3 Ethernet

ISO 3309 HDLC

RFC1661 PPP (ex 1548)

RFC1662 PPP in HDLC framing (ex 1549)

RFC2615 PoS (ex 1619)

G.707 SDH (especially the new section 11 – VCAT)

G.709 OTN

G.7041 GFP

G.7042 LCAS for SDH

G.7043 VCAT for PDH

X.85 IP over SDH using LAPS

X.86 Ethernet over SDH using LAPS

Y(J)S EoS Slide 11

BackgroundBackground

EthernetEthernet

Y(J)S EoS Slide 12

Ethernet frameEthernet frame

For our purposes, “Ethernet” is any layer 2 protocol using 1 of the following frame formats :

DA (6B) SA (6B) T/L (2B) data (0-1500B) pad (0-46) FCS (4B)

64 – 1518 B

DA(6B) SA(6B) T/L(2B) data (0-1500B) pad(0-46) FCS(4B)VT(2B) VLAN(2B)

68 – 1522 B

Y(J)S EoS Slide 13

Ethernet frame sizeEthernet frame size

Minimum frame is 64 bytes

Maximum payload was 1500 bytes – and maximum frame was 1522 bytes

802.3as lengthened maximum frame to 2000 bytes

Various physical layer modulations and framing

Rates : 10 Mbps, 100 Mbps, 1 Gbps, 10 Gbps, …

Y(J)S EoS Slide 14

BackgroundBackground

HDLCHDLC

Y(J)S EoS Slide 15

Packet to bit streamPacket to bit streamThe first problem in converting Ethernet to TDM: Ethernet consists of frames carrying packets TDM is a continuous bit stream

We can convert a sequence of packets into a bit stream by using an “idle code”

For example, we can use a sequence of 1s as idle indication

The appearance of a 0 bit indicates that data follows

packet 1 packet 2 packet 3 packet 4

packet 1 packet 2 packet 3 packet 4

111111111111111111111110 packet 1 0111111111111111111110 packet 2 011111111111111111111110 01111110 packet 3 01111111111111111

Y(J)S EoS Slide 16

Packet to bit stream Packet to bit stream (cont.)(cont.)

How does the receiver know when to return to idle?

We use a specific “flag” (HDLC uses hex 7E = 01111110)

We can use the flag as the idle code as well

Some implementations allow “zero sharing”

But the flag must not appear in valid data!

If we have access to the physical layer we can mark there (“violations”)

Otherwise (we only access bits) we must disallow the idle code

by replacing it with something else

01111110 01111110 01111110 packet 1 01111110 01111110 01111110 packet 2 01111110 01111110 01111110 01111110 packet 3 01111110

0111111011111101111110 packet 1 011111101111110 01111110 packet 2 011111101111110 1111110 1111110 packet 3 011111101111110

Y(J)S EoS Slide 17

HDLC flagsHDLC flagsISO developed High level Data Link C based on IBM’s SDLC

HDLC inputs packets of bytes

HDLC uses hex 7E as its idle code (“flag”) 01111110

So an idle HDLC stream repeats 7E

Alternatively, 1s can be sent as idle, flags as delineators

There are two methods of disallowing flags

bit stuffing (zero insertion)

byte (octet) stuffing

01111110 01111110 01111110 packet 1 01111110 01111110 01111110 packet 2 01111110 01111110 01111110 01111110 packet 3 01111110

11111111111111111 01111110 packet 1 01111110 111111111101111110 packet 2 01111110 11111111111111111101111110 packet 3 01111110

Y(J)S EoS Slide 18

Bit stuffing / zero insertionBit stuffing / zero insertion

ECMA-40

Whenever the encoder sees 5 successive 1s it appends a 0thus there are never 6 successive 1s in the data

When the decoder sees 5 successive 1s : If the next bit is a 0 it is deleted If the next bit is a 1 then this is the closing flag

Notes: bit stream length is no longer necessarily divisible by 8 bit stream length is not a priori predictable worst case expansion is 20% encoding/decoding is easy in HW, hard in SW

Y(J)S EoS Slide 19

Byte (octet) stuffingByte (octet) stuffingRFC1549

Whenever the encoder sees hex 7E It replaces it with 7D 5E

Whenever the encoder sees hex 7DIt replaces it with 7D 5D

Optionally other codes (e.g. some under hex 20) can be “escaped”Second byte is original with 6th bit complemented (xor with hex 20)e.g. ^Q = hex 11→ 7D 31 ^S = hex 13 → 7D 33

When the receiver sees 7D xx It replaces it with the original byte (complementing 6th bit)

Notes: bit stream remains byte oriented length expansion is typically about 1%, but can range from 0 to 100% ! (there is also a consistent overhead algorithm – but not in use) encoding/decoding is easy in SW

Y(J)S EoS Slide 20

HDLC framingHDLC framing

HDLC frame is bounded by flags, and has a particular structure

Many variants (SDLC, ISO, LAPB, LAPD, LAPF, LAPS, SS7, PPP-HDLC, Cisco-HDLC, etc)

Address: There may be no address (e.g. SS7 HDLC) SDLC always had 8 bit addresses ISO 3309 HDLC has structured multibyte address

– Service Access Point Identifier (MSB of SAPI =1 may indicate broadcast/multicast)

– EA=1 means 8 bit, EA=0 means extended address– C/R=1 for commands, C/R=0 for responses

The single byte hex FF is recognized as the broadcast address

flag (8) flag (8)address (0/8/16) ctrl (8/16) data FCS (16/32)

EAC/RSAPI EA

Y(J)S EoS Slide 21

HDLC controlHDLC control

HDLC networks can be configured: Balanced – all stations have equal responsibility Unbalanced – primary and one or more secondary stations

and HDLC can operate : Best effort (datagram)

– uses Un-numbered (U) frames Reliable (Asynchronous Balanced Mode)

– uses frames with sequence numbers in control field Information (I) frames (data + acknowledgement) Supervisory (S) frames (only acknowledgement)

The various frame types are indicated by the control fieldwhich varies widely between different protocols

Y(J)S EoS Slide 22

HDLC FCSHDLC FCS

HDLC uses a Frame Check Sequence to detect errors

The FCS is implemented as a shift-register

CRC-16 X16 + X12 + X5 + 1 CRC-32 X32 + X26 + X23 + X22 + X16 + X12 + X11 + X10 + X8 + X7 + X5 + X4 + X2 + X + 1

Some HDLC-based protocols require 32 bit FCSothers allow 16 bit but recommend 32 bit FCS

Y(J)S EoS Slide 23

BackgroundBackground

PPPPPP

Y(J)S EoS Slide 24

Point to Point Protocol (RFC 1661)Point to Point Protocol (RFC 1661)PPP is a method for transporting datagrams between 2 peers

over full-duplex, point-to-point data links – for example: short lines, leased lines, dial-up modems

PPP may be used to connect hosts to routers, and routers to routers

PPP is made up of 3 components:

encapsulation method for (multiprotocol) datagrams

Link Control Protocol for establishing, configuring, and testing data-link connections

Network Control Protocols for establishing and configuring different network-layer protocols

PPP is a suite containing many protocolsML-PPP, PPPoE, BAP, BCP, IPCP, …

Y(J)S EoS Slide 25

Basic PPP encapsulation (RFC 1661)Basic PPP encapsulation (RFC 1661)

Encapsulation enables demuxing of different network-layer protocols

Only 1 field needs to be examined for protocol determination

Protocol field obeys ISO 3309 rules:

– protocol value must be odd (for EA=1)

– if 16-bit, then the LSB of first byte must be zero (for EA=0)

PPP protocol values managed by IANA

(http://www.iana.org/assignments/ppp-numbers)

Padding may be used (e.g. to cause header to fall on 32-bit boundary)

protocol (8/16) information padding

Y(J)S EoS Slide 26

PPP using HDLC framing (RFC 1662)PPP using HDLC framing (RFC 1662)

When using PPP over synchronous links we use HDLC-like framing

1 byte Broadcast address is used by default (users may define alternative address)

Synchronous Link may be bit-oriented or byte-oriented

Basic PPP encapsulation is extended by 8 bytes

Bit stuffing or byte stuffing allowed

Escape mechanism allows transparent transfer of control data (e.g. ^S/^Q)

enables removal of spurious control data (inserted by intermediate boxes)

flag

7E

address

FF

ctrl

03

information FCS

(16/32b)

protocol

(8/16b)

padding

(optional)

flag

7E

Y(J)S EoS Slide 27

RFC1662 vs. X.85RFC1662 vs. X.85

ITU-T X.85 defines IP over SDH using LAPS (will study later)

Its encapsulation is similar to RFC1662 (but can’t co-exist with it)

Instead of the protocol ID it has a SAPI = 21 for IPv4 =57 for IPv6

The FCS MUST be 32 bits and no padding is used

No special escaping is defined

flag

7E

address

04

ctrl

03

IP Packet FCS

(32b)

SAPI

(16b)

flag

7E

flag

7E

address

FF

ctrl

03

information FCS

(16/32b)

protocol

(8/16b)

padding

(optional)

flag

7E1662

X.85

PPP frame

Y(J)S EoS Slide 28

BackgroundBackground

SONET/SDHSONET/SDH

Note:

For more information – see SONET/SDH course.

Y(J)S EoS Slide 29

SONET architectureSONET architecture

SONET (SDH) has at 3 layers: path – end-to-end data connection, muxes tributary signals path section

– there are STS paths + Virtual Tributary (VT) paths

line – protected multiplexed SONET payload multiplex section section – physical link between adjacent elements regenerator section

Each layer has its own overhead to support needed functionality

SDH terminology

Path

Termination

Path

Termination

Line

Termination

Line

Termination

Section

Termination

path

line line line

ADM ADMregenerator

section section sectionsection

Y(J)S EoS Slide 30

SONET STS-1 frameSONET STS-1 frame

Synchronous Transfer Signals are bit-signals (OC are optical)

Each STS-1 frame is 90 columns * 9 rows = 810 bytes

There are 8000 STS-1 frames per secondso each byte represents 64 kbps (each column is 576 kbps)

Thus the basic STS-1 rate is 51.840 Mbps

90 columns

9 ro

ws

Y(J)S EoS Slide 31

SDH STM-1 frameSDH STM-1 frame

Synchronous Transport Modules are the bit-signals for SDH

Each STM-1 frame is 270 columns * 9 rows = 2430 bytes

There are 8000 STM-1 frames per second

Thus the basic STM-1 rate is 155.520 Mbps

3 times the STS-1 rate!

270 columns

9 ro

ws

…

Y(J)S EoS Slide 32

SONET/SDH ratesSONET/SDH rates

STS-N has 90N columns STM-M corresponds to STS-N with N = 3M

SDH rates increase by factors of 4 each time

STS/STM signals can carry PDH tributaries, for example:

STS-1 can carry 1 T3 or 28 T1s or 1 E3 or 21 E1s

STM-1 can carry 3 E3s or 63 E1s or 3 T3s or 84 T1s

SONET SDH columns rate

STS-1 90 51.84M

STS-3 STM-1 270 155.52M

STS-12 STM-4 1080 622.080M

STS-48 STM-16 4320 2488.32M

STS-192 STM-64 17280 9953.28M

Y(J)S EoS Slide 33

SONET/SDH tributariesSONET/SDH tributaries

E3 and T3 are carried as Higher Order Paths (HOPs)

E1 and T1 are carried as Lower Order Paths (LOPs)

SONET SDH T1 T3 E1 E3 E4

STS-1 28 1 21 1

STS-3 STM-1 84 3 63 3 1

STS-12 STM-4 336 12 252 12 4

STS-48 STM-16 1344 48 1008 48 16

STS-192 STM-64 5376 192 4032 192 64

Y(J)S EoS Slide 34

Synchronous Payload Envelope

STS-1 frame structureSTS-1 frame structure9

row

s

section + lineoverhead

6 ro

ws

3 ro

ws

Section overhead is 3 rows * 3 columns = 9 bytes = 576 kbpsframing, performance monitoring, management

Line overhead is 6 rows * 3 columns = 18 bytes = 1152 kbpsprotection switching, line maintenance, mux/concat, SPE pointer

SPE is 9 rows * 87 columns = 783 bytes = 50.112 Mbps

Similarly, STM-1 has 9 (different) columns of section+line overhead !

90 columns

9 ro

ws

Y(J)S EoS Slide 35

STM-1 frame structureSTM-1 frame structure

TransportOverhead

TOHSimilarly, STM-1 has 9 (different) columns of transport overhead !

RS overhead is 3 rows * 9 columns

Pointer overhead is 1 row * 9 columns

MS overhead is 5 rows * 9 columns

SPE is 9 rows * 87 columns

…

270 columns

Y(J)S EoS Slide 36

ScramblingScramblingSONET/SDH receivers recover clock based on incoming signal

Insufficient number of 0-1 transitions causes degradation of clock performance

In order to guarantee sufficient transitions, SONET/SDH employ a scrambler All data except first row of section overhead is scrambled Scrambler is 7 bit self-synchronizing X7 + X6 + 1 Scrambler is initialized with ones

A short scrambler is sufficient for voice data

but NOT for data which may contain long stretches of zeros

When sending data an additional payload scrambler is used modern standards use 43 bit X43 + 1 run continuously on ATM payload bytes (suspended for 5 bytes of cell tax) run continuously on HDLC payloads

Z-43

Xn Yn = Xn + Yn-43

Y(J)S EoS Slide 37

HOP SPE structureHOP SPE structure

2 bytes in the line overhead point to the STS path overhead POHpointer (floating) allows frequency/phase compensation

(after re-arranging) POH is one column of 9 rows (9 bytes = 576 kbps)

Y(J)S EoS Slide 38

Path overheadPath overhead

POH is responsible for – path performance monitoring– status (including of mapped payloads)

– trace

2 bytes are of particular interest to us:

C2 is the “signal label” indicates path payload type

H4 is the “multiframe indication” used by VCAT/LCAS (discussed later)

J1

B3

C2

G1

F2

H4

F3

K3

N1

POH

C2 (hex)

Payload type

00 unequipped

01 nonspecific

02 LOP (TUG)

04 E3/T3

12 E4

13 ATM

16 PoS – RFC 1662

18 LAPS X.85

1A 10G Ethernet

1B GFP

CF PoS - RFC1619

Y(J)S EoS Slide 39

STS-1 HOPSTS-1 HOP

1 column of SPE is POH

2 more (“fixed stuffing”) columns are reserved

We are left with84 columns = 756 bytes = 48.384 Mbps for payload

This is enough for a E3 (34.368M) or a T3 (44.736M)

1 875930

Y(J)S EoS Slide 40

LOPLOP

To carry lower rate payloads, divide 84 available columns into 7 * 12 interleaved columns, i.e. 7 Virtual Tributary (VT) groups

VT group is 12 columns of 9 rows, i.e. 108 bytes or 6.912 Mbps

VT group is composed of VT(s) There are different types of VT in order to carry different types of payload all VTs in VT group must be of the same type but different VT groups in same SPE can have different VT types

A VT can have 3, 4, 6 or 12 columns

1 875930 1 2 3 4 5 6 7VTG

Y(J)S EoS Slide 41

SONET/SDH : VT/VC typesSONET/SDH : VT/VC types

VT/STS VC column

rate

payload

VT 1.5 VC-11 3 1.728 DS1 (1.544)

VT 2 VC-12 4 2.304 E1 (2.048)

VT 3 6 3.456 DS1C (3.152)

VT 6 VC-2 12 6.912 DS2 (6.312)

STS-1 VC-3 48.384 E3 (34.368)

STS-1 VC-3 48.384 DS3 (44.736)

STS-3c VC-4 149.760 E4 (139.264)

LOP

HOP

standard PDH rates map efficiently into SONET/SDH !

4 per group

3 per group

2 per group

1 per group

Y(J)S EoS Slide 42

Payload capacityPayload capacity

VT1.5/VC-11 has 3 columns = 27 bytes = 1.728 Mbps

but 2 bytes are used for overhead

so actually only 25 bytes = 1.6 Mbps are available

Similarly

VT2/VC-12 has 4 columns = 36 bytes = 2.304 Mbps

but 2 bytes are used for overhead

So actually only 34 bytes = 2.176 Mbps are available

Y(J)S EoS Slide 43

VCATVCAT

Virtual ConcatenationVirtual Concatenation

Y(J)S EoS Slide 44

ConcatenationConcatenation

Payloads that don’t fit into standard VT/VC sizes can be accommodatedby concatenating of several VTs / VCs

For example, 10 Mbps doesn’t fit into any VT or VCso w/o concatenation we need to put it into an STS-1 (48.384 Mbps)the remaining 38.384 Mbps can not be used

We would like to be able to divide the 10 Mbps among 7 VT1.5/VC-11 s = 7 * 1.600 = 11.20 Mbps or5 VT2/VC-12 s = 5 * 2.176 = 10.88 Mbps

Y(J)S EoS Slide 45

ConcatenationConcatenationThere are 2 ways to concatenate X VTs or VCs:

Contiguous Concatenation (G.707 11.1)

– HOP – STS-Nc (SONET) or VC-4-Nc (SDH)

or LOP – 1-7 VC-2-Nc into a VC-3– since has to fit into SONET/SDH payload

only STS-Nc : N=3 * 4n or VC-4-Nc : N=4n

– components transported together and in-phase– requires support at intermediate network elements

Virtual Concatenation (VCAT G.707 11.2) – HOP – STS-1-Xv or STS-Nc-Xv (SONET) or VC-3/4-Xv (SDH)

or LOP – VT-1.5/2/3/6-Xv (SONET) or VC-11/12/2-Xv (SDH)

– HOP: X ≤ 256 LOP: X ≤ 64 (limitation due to bits in header)

– payload split over multiple STSs / STMs– fragments may follow different routes– requires support only at path terminations– requires buffering and differential delay alignment

Y(J)S EoS Slide 46

Contiguous Concatenation: STS-3cContiguous Concatenation: STS-3c270 columns

9 ro

ws …

9 columns of section and

line overhead

3 columns of path overhead

258 columns of SPE STS-3

270 columns

9 ro

ws …

9 columns of section and

line overhead

1 column of path overhead

260 columns of SPE STS-3c

258 columns * 0.576 = 148.608 Mbps

260 columns * 0.576 = 149.760 Mbps

Y(J)S EoS Slide 47

STS-N vs. STS-NcSTS-N vs. STS-Nc

Although both have raw rates of 155.520 Mbps

STS-3c has 2 more columns (1.152Mbps) available

More generally, For STS-Nc gains (N-1) columnse.g. STS-12c gains 11 columns = 6.336Mbps vis a vis STS-12STS-48c gains 47 columns = 27.072 MbpsSTS-192c gains 191 columns = 110.016 Mbps !

However, an STS-Nc signal is not as easily separablewhen we want to add/drop component signals

Y(J)S EoS Slide 48

Virtual ConcatenationVirtual Concatenation

VCAT is an inverse multiplexing mechanism (round-robin)VCAT members may travel along different routes in SONET/SDH network

Intermediate network elements don’t need to know about VCAT(unlike contiguous concatenation that is handled by all intermediate nodes)

…

H4

Y(J)S EoS Slide 49

SDH virtually concatenated VCsSDH virtually concatenated VCs

So we have many permissible rates

1.600, 2.176, 3.200, 4.352, 4.800, 6.400, 6.528, 6.784, 8.000, …

VC Capacity (Mbps) if all members in one VC

VC-11-Xv 1.600, 3.200, …

1.600X

in VC-3 X ≤ 28 C ≤ 44.800

in VC-4 X ≤ 64 C ≤ 102.400

VC-12-Xv 2.176, 4.352, …

2.176X

in VC-3 X ≤ 21 C ≤ 45.696

in VC-4 X ≤ 63 C ≤ 137.088

VC-2-Xv 6.784, 13.568, …,

6.784X

in VC-3 X ≤ 7 C ≤ 47.448

in VC-4 X ≤ 21 C ≤ 142.464

Y(J)S EoS Slide 50

SONET virtually concatenated VTsSONET virtually concatenated VTs

VT Capacity (Mbps) If all members in one STS

VT1.5-Xv 1.600, 3.200, … 1.600X in STS-1 X ≤ 28 C ≤ 44.800

in STS-3c X ≤ 64 C ≤ 102.400

VT2-Xv 2.176, 4.352, … 2.176X in STS-1 X ≤ 21 C ≤ 45.696

in STS-3c X ≤ 63 C ≤ 137.088

VT3-Xv 3.328, 6.656, … 3.328X in STS-1 X ≤ 14 C ≤ 46.592

in STS-3c X ≤ 42 C ≤ 139.776

VT6-Xv 6.784, 13.568, … 6.784X in STS-1 X ≤ 7 C ≤ 47.448

in STS-3c X ≤ 21 C ≤ 142.464

So we have many permissible rates

1.600, 2.176, 3.200, 3.328, 4.352, 4.800, 6.400, 6.528, 6.656, 6.784, …

Y(J)S EoS Slide 51

Efficiency comparisonEfficiency comparison

Using VCAT increases efficiency to close to 100% !

rate w/o VCAT efficiency with VCAT efficiency

10 STS-1 21% VT2-5v

VC-12-5v

92%

100 STS-3c

VC-4

67% STS-1-2v

VC-3-2v

100%

1000 STS-48c

VC-4-16c

42% STS-3c-7v

VC-4-7v

95%

Y(J)S EoS Slide 52

PDH VCATPDH VCAT

Recently ITU-T G.7043 expanded VCAT to E1,T1,E3,T3

Enables bonding of up to 16 PDH signals to support higher rates

Only bonding of like PDH signals allowed (e.g. can’t mix E1s and T1s)

Multiframe is always per G.704/G.832 (e.g. T1 – ESF 24 frames, E1 16 frames)

1 byte per multiframe is VCAT overhead (SQ, MFI, MST, CRC)

Supports LCAS (to be discussed next)

TS0

1st frameof4 E1s

VCAToverhead

octet

timeeach E1

Y(J)S EoS Slide 53

PDH VCAT overhead octetPDH VCAT overhead octet

There is one VCAT overhead octet per multiframe, so net rate is

T1: (24*24-1=) 575 data bytes per 3 ms. multiframe = 191.666 kB/s

E1: (16*30-1=) 495 data bytes per 2 ms multiframe = 247.5 kB/s

T3 and E3 can also be used

We will show the overhead octet format later

(when using LCAS, the overhead octet is called VLI)

TS0

frames of an E1

VCAToverhead

octet

…

Y(J)S EoS Slide 54

Delay compensationDelay compensation

802.1ad Ethernet link aggregation cheats– each identifiable flow is restricted to one link– doesn’t work if single high-BW flow

VCAT is completely general– works even with a single flow

VCG members may travel over completely separate pathsso the VCAT mechanism must compensate for differential delay

Requirement for over ½ second compensation

Must compensate to the bit level

but since frames have Frame Alignment Signalthe VCAT mechanism only needs to identify individual frames

Y(J)S EoS Slide 55

VCAT bufferingVCAT buffering

Since VCAT components may take different paths

At egress the members are no longer in the proper temporal relationship

VCAT path termination function buffers membersand outputs in proper order (relying on POH sequencing)(up to 512 ms of differential delay can be tolerated)

VCAT defines a multiframe to enable delay compensation– length of multiframe determines delay that can be accommodated

H4 byte in member’s POH contains : sequence indicator (identifies component) (number of bits limits X) MFI multiframe indicator (multiframe sequencing to find differential delay)

Y(J)S EoS Slide 56

Multiframes and superframesMultiframes and superframes

Here is how we compensate for 512 ms of differential delay

512 ms corresponds to a superframe is 4096 TDM frames (4096*0.125m=512m)

For HOS SDH VCAT and PDH VCAT (H4 byte or PDH VCAT overhead)

The basic multiframe is 16 frames

So we need 256 multiframes in a superframe (256*16=4096)

The MultiFrame Indicator is divided into two parts: MFI1 (4 bits) appears once per frame

– and counts from 0 to 15 to sequence the multiframe MFI2 (8bits) appears once per multiframe

– and counts from 0 to 255

For LOS SDH (bit 2 of K4 byte)– a 32 bit frame is built and a 5-bit MFI is dedicated– 32 multiframes of 16 ms give the needed 512 ms

Y(J)S EoS Slide 57

LCASLCAS

Link Capacity Adjustment SchemeLink Capacity Adjustment Scheme

Y(J)S EoS Slide 58

LCASLCAS

LCAS is defined in G.7042 (also numbered Y.1305)

LCAS extends VCAT by allowing dynamic BW changes

LCAS is a protocol for dynamic adding/removing of VCAT members – hitless BW modification– similar to Link Aggregation Control Protocol for Ethernet links

LCAS is not a “control plane” or “management” protocol– it doesn’t allocate the members– still need control protocols to perform actual allocation

LCAS is a “handshake” protocol– it enables the path ends to negotiate the additional / deletion – it guarantees that there will be no loss of data during change– it can determine that a proposed member is ill suited– it allows automatic removal of faulty member

Y(J)S EoS Slide 59

LCAS – how does it work?LCAS – how does it work?LCAS is unidirectional (for symmetric BW need to perform twice)

LCAS functions can be initiated by source or sink

LCAS assumes that all VCG members are error-free

– LCAS messages are CRC protected

LCAS messages are sent in advance – sink processes messages after differential compensation– message describes link state at time of next message– receiver can switch to new configuration in time

LCAS messages are in the upper nibble of– H4 byte for HOS SONET/SDH– K4 byte for LOS SONET/SDH– VCAT overhead octet for PDH – VCAT and LCAS Information

LCAS messages employ redundancy– messages from source to sink are member specific– messages from sink to source are replicated

J1

B3

C2

G1

F2

H4

F3

K3

N1

POH

Y(J)S EoS Slide 60

LCAS control messagesLCAS control messages

LCAS adds fields to the basic VCAT ones

Fields in messages from source to sink:– MFI MultiFrame Indicator– SQ SeQuence indicator (member ID inside VCAT group)– CTRL ConTRoL (IDLE, being ADDed, NORMal, End of Sequence, Do Not Use)

– GID Group Identification (identifies VCAT group)

Fields in messages from sink to source (identical in all members):– MST Member Status (1 bit for each VCG member)– RS-Ack ReSequence Acknowledgement

Fields in both directions– CRC Cyclic Redundancy Code

The precise format depends on the VCAT type (H4, K4, PDH)

Note: for H4 format SQ is 8 bits, so up to 256 VCG members

for PDH SQ is only 4 bits, so up to 16 VCG members

Y(J)S EoS Slide 61

H4 formatH4 format

MFI2 bits 1-4 0 0 0 0 MFI2 bits 5-8 0 0 0 1

CTRL 0 0 1 0 0 0 0 GID 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1

CRC-8 bits 1-4 0 1 1 0CRC-8 bits 5-8 0 1 1 1

MST bits 1 0 0 0more MST bits 1 0 0 1

0 0 0 RS-ACK 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 1

SQ bits 1-4 1 1 1 0SQ bits 5-8 1 1 1 1

16 frame m

ultiframeMFI1

rese

rved

fie

lds

rese

rved

fie

lds

Y(J)S EoS Slide 62

H4 format – some commentsH4 format – some comments

CRC-8 (when using K4 it is CRC-3)– covers the previous 14 frames (not sync’ed on multiframe)– polynomial x8 + x2 + x + 1

MST– each VCG member carries the status of all members– so we need 256 bits of member status– this is done by muxing MST bits– there are MST bits per multiframe– and 32 multiframes in an MST multiframe– no special sequencing, just MFI2 multiframe mod 32

GID– single bit - cycles through 215-1 LFSR sequence

Y(J)S EoS Slide 63

VLI formatVLI format

MFI2 bits 1-4 0 0 0 0 MFI2 bits 5-8 0 0 0 1

CTRL 0 0 1 0 0 0 0 GID 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1

CRC-8 bits 1-4 0 1 1 0CRC-8 bits 5-8 0 1 1 1

MST bits 1 0 0 0more MST bits 1 0 0 1

0 0 0 RS-ACK 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 1 1 0 1 0 0 0 0 1 1 1 0

SQ 1 1 1 1

16 frame m

ultiframeMFI1

rese

rved

fie

lds

rese

rved

fie

lds

Y(J)S EoS Slide 64

LCAS – adding a member (1)LCAS – adding a member (1)When more/less BW is needed, we need to add/remove VCAT members

Adding/removing VCAT members first requires provisioning (management)

LCAS handles member sequence numbers assignment

LCAS ensures service is not disrupted

Example: to add a 4th member to group “1”

Initial state:

Step 1: NMS provisions new member

source sends CTRL=IDLE for new member

sink sends MST=FAIL for new member

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=EOS

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=EOS

GID=g SQ=FF CTRL=IDLE

Y(J)S EoS Slide 65

LCAS – adding a member (2)LCAS – adding a member (2)Step 2: source sends CTRL=ADD and SQ

sink sends MST=OK for new member if it has been provisioned if receiving new member OK if it is able to compensate for delay

otherwise it will send MST=FAILand source reports this to NMS

Step 3: source sends CTRL=EOS for new member

new member starts to carry traffic

sink sends RS-ACK

Note 1: several new members may be added at once

Note 2: removing a member is similar

Source puts CTRL=IDLE for member to be removed and stops using it

All member sequence numbers must be adjusted

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=EOS

GID=g SQ=4 CTRL=ADD

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=NORM

GID=g SQ=4 CTRL=EOS

Y(J)S EoS Slide 66

LCAS – service preservationLCAS – service preservationTo preserve service integrity if sink detects a failure of a VCAT member

LCAS can temporarily remove member (if service can tolerate BW reduction)

Example: Initial state

Step 1: sink sends MST=FAIL for member 2 source sends CTRL=DNU (special treatment if EoS) and ceases to use member 2Note: if EoS fails, renumber to ensure EoS is active

Step 2: sink sends MST=OK indicating defect is cleared source returns CTRL to NORM and starts using the member again Note: if NMS decides to permanently remove the member, proceed as in previous slide

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=NORM

GID=g SQ=4 CTRL=EOS

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=DNU

GID=g SQ=3 CTRL=NORM

GID=g SQ=4 CTRL=EOS

Y(J)S EoS Slide 67

PoSPoS

Packet over SONETPacket over SONET

Y(J)S EoS Slide 68

Packet over SONETPacket over SONET

Currently defined in RFC2615 (PPP over SONET) obsoletes RFC1619

SONET/SDH path can provide a point-to-point byte-oriented full-duplex synchronous link

PPP is ideal for data transport over such a link

PoS uses PPP in HDLC framing to provide a byte-oriented interfaceto the SONET/SDH infrastructure

SONET/SDH POH signal label (C2) indicates PoS as C2=16 (C2=CF if no scrambler)

Y(J)S EoS Slide 69

PoS architecturePoS architecture

PoS is based on PPP in HDLC framing

Since SONET/SDH is byte oriented, byte stuffing is employed

A special scrambler is used to protect SONET/SDH timing

PoS operates on IP packets

If IP is delivered over Ethernet– the Ethernet is terminated (frame removed)– Ethernet must be reconstituted at the far end– require routers at edges of SONET/SDH network

IP

PPP

HDLC

SONET/SDH

Y(J)S EoS Slide 70

What happened to the Ethernet ?What happened to the Ethernet ?

The conventional model:

Ethernet is a LAN technology – last 100m– 10s of hosts

IP is a WAN technology– data transported in native IP– different L2 technologies for last segment

But modern Ethernet wants to be more

IPEthernet Ethernet

Y(J)S EoS Slide 71

PoS DetailsPoS Details

IP packet is encapsulated in PPP– default MTU is 1500 bytes– up to 64,000 bytes allowed if negotiated by PPP

FCS is generated and appended

PPP in HDLC framing with byte stuffing

43 bit scrambler is run over the SPE

byte stream is placed octet-aligned in SPE– (e.g. 149.760 Mbps of STM-1)– HDLC frames may cross SPE boundaries

Y(J)S EoS Slide 72

RFC2615 vs. RFC1619RFC2615 vs. RFC1619

RFC1619 did not have the 43 bit scrambler

Malicious users could generate packets containing frame alignment pattern

– deceiving framer into mis-syncing with low transition density

– degrading clock performance containing SONET/SDH reset scrambler pattern

– causing errors

So RFC2615 added the scramblerscrambler does not reset during usehard to guess proper internal state

Y(J)S EoS Slide 73

POS problemsPOS problems

PoS is BW efficient

but POS has its disadvantages

BW must be predetermined

HDLC BW expansion and nondeterminacy

BW allocation is tightly constrained by SONET/SDH capacities

– e.g. GbE requires a full OC-48 pipe

POS requires removing the Ethernet headers

– So lose RPR, VLAN, 802.1p, multicasting, etc

POS requires IP routers

Y(J)S EoS Slide 74

LAPSLAPS

Link Access Protocol over SDHLink Access Protocol over SDH

X.85 and X.86X.85 and X.86

Y(J)S EoS Slide 75

LAPSLAPS

In 2001 ITU-T introduced protocols for transporting packets over SDH

X.85 IP over SDH using LAPS

X.86 Ethernet over LAPS

Built on series of ITU “LAPx” HDLC-based protocols

Use ISO HDLC format

Implement connectionless byte-oriented protocols over SDH

X.85 is very close to (but not quite) IETF PoS

Y(J)S EoS Slide 76

X.85 vs. X.86X.85 vs. X.86

X.85 transports IP packets if delivered over Ethernet, the Ethernet is terminated

X.86 transports Ethernetcan transport all sorts of Ethernet traffic – not only IP packets

IP

LLC

MAC

IP

LLC

MAC

IP

LLC

MAC

LAPS

SDH

X.86

IP

LLC

MAC

IP

LLC

MAC

IP

LAPS

SDH

X.85

Y(J)S EoS Slide 77

X.85X.85

IP over SDH using LAPS

address = 04 (or FF for compatibility with PoS)

SAPI = 21 for IPv4 =57 for IPv6 (changed to be like PoS)

Scrambler always used

Can use LOP VCs, HOP VCs or STMs

flag

7E

address

(16b)

ctrl

03

IP Packet FCS

(32b)

SAPI

(16b)

flag

7E

Y(J)S EoS Slide 78

X.86X.86

Similar to X.85 (IP over SDH using LAPS)but transports the entire Ethernet frame

Provides a virtual MII/GMII interface

Transparent to all Ethernet features (VLAN, P bits, RPR, etc.)

Rate adaptation by adding hex DD (after byte stuffing 7D DD)

Ammendment specifies use of Ethernet PAUSE frames for rate limiting

MAC

LAPS

SDH

reconciliation

rate adaptation

MII/GMII

flag

7E

address

(16b)

ctrl

03

Ethernet frame

DA SA T/L INFO PAD FCS

FCS

(32b)

SAPI

FE01

flag

7E

Y(J)S EoS Slide 79

LAPS drawbacksLAPS drawbacks

Only IP or Ethernet payloads

Single bit errors (e.g. in flags) may cause misalignment

Not very efficient

HDLC BW expansion

HDLC BW nondeterminacy

Y(J)S EoS Slide 80

GFPGFP

Generic Framing ProcedureGeneric Framing Procedure

Y(J)S EoS Slide 81

GFP architectureGFP architectureDefined in ITU-T G.7041 (also numbered Y.1303)

originally developed in T1X1 to fix ATM limitations(like ATM) uses HEC protected frames instead of HDLC

GFP generically encapsulates client (e.g. IP, Ethernet)onto transport network (e.g. SONET/SDH, OTN)

Client may be PDU-oriented (Ethernet MAC, IP) or block-oriented (GbE, fiber channel)

GFP frames– are octet aligned– contain at most 65,535 bytes– consist of a header + payload area

Any idle time between GFP frames is filled with GFP idle frames

Ethernet IP other

GFP – client specific part

GFP – common part

SDH OTN other

HDLC

PDH

Y(J)S EoS Slide 82

GFP frame structureGFP frame structure

Every GFP frame has a 4-byte core header– 2 byte Payload Length Indicator PLI = 01,2,3 are for control frames

– 2 byte core Header Error Control X16 + X12 + X5 + 1

– entire core header is XOR’ed with B6AB31E0 so idle frames are B6AB31E0 (Barker-like codes)

Idle GFP frames – have PLI=0 – have no payload area

Non-idle GFP frames – have ≥ 4 bytes in payload area– the payload has its own header– 2 payload modes : GFP-F and GFP-T– optionally protect payload with CRC-32– payload is scrambled like PoS

PLI (2B)

cHEC (2B)

payload header (4-64B)

payload

optional payloadFCS (4B)

coreheader

payloadarea

Y(J)S EoS Slide 83

GFP payload headerGFP payload header

GFP payload header has– type (2B)– type HEC (CRC-16)– extension header (0-60B)

either null or linear extension (payload type muxing)

– extension HEC (CRC-16)

type consists of– Payload Type Identifier (3b)

PTI=000 for client data PTI=100 for client management (OAM dLOS, dLOF)

– Payload FCS Indicator (1b) PFI=1 means there is a payload FCS

– Extension Header ID (4b)– User Payload Identifier (8b)

values for Ethernet, IP, PPP, FC, RPR, MPLS, etc.

type (2B)

tHEC (2B)

extension header (0-58B)

eHEC (2B)

UPI (8b)

PTI (3b) EXI (4b)PFI

Y(J)S EoS Slide 84

GFP modes GFP modes

GFP-F - frame mapped GFP

Good for PDU-based protocols (Ethernet, IP, MPLS)or HDLC-based ones (PPP)

Client PDU is placed in GFP payload field

GFP-T – transparent GFP

Good for protocols that exploit physical layer capabilities

In particular8B/10B line code used in fiber channel, GbE, FICON, ESCON, DVB, etc

Were we to use GFP-F would lose control info, GFP-T is transparent to these codes

Also, GFP-T needn’t wait for entire PDU to be received (adding delay!)

Y(J)S EoS Slide 85

GFP-T GFP-T Main application – Storage Area Networks (SAN)SANs use 8B/10B line code and are very delay sensitive

8B/10B line code maps each of the 256 values of the 8-bit inputinto 1 or 2 different 10 bit wordsMaintains a running 0-1 balance and when encoding an input with 2 possibilities, it

chooses the one that improves the balance

spare 10b symbols are used as control codes (e.g. start/end of frame)

Were we to use GFP-F would lose control info, GFP-T is transparent to these codes

Also, GFP-T needn’t wait for entire PDU to be received (adding delay!)

GFP-T maps 8B/10B line code into 64B/65B block code

Y(J)S EoS Slide 86

GFP-F GFP-F

Client packet/frame without un-needed overhead (e.g. flags, preamble, etc)

is placed in GFP payload field

Interface is at link layer

More BW efficient than GFP-T since idle periods are filtered outpreambles, frame-start, etc are also not transported

GFP-F must know the client protocol in order to detect frames

Can mux different client protocols on a frame to frame basis

If the client protocol has a good FCS, don’t need to use GFP’s FCS

GFP-F is used for EoS

Either IP in PPP or native Ethernet can be used

Y(J)S EoS Slide 87

GFP advantagesGFP advantages

Supports multiple protocols (not just Ethernet and IP)

For Ethernet, GFP can transparently transport entire frame

Robust – single bit errors do not cause loss of alignment

Constant predictable overhead

Good efficiency (similar to LAPS best case)

GFP-T for SAN support

Can run over OTN (G.709) as well as SONET

Y(J)S EoS Slide 88

AlternativesAlternatives

Y(J)S EoS Slide 89

There are yet other ways …There are yet other ways …

Ethernet in the first mile (EFM)

WAN-PHY (10GBASE-W)

Ethernet over wavelengths (EoW) or OTN (G.709)

Ethernet over Resilient Packet Rings (RPR)

Ethernet pseudowires (PWs)

Y(J)S EoS Slide 90

Ethernet in the First MileEthernet in the First MileIEEE 802.3ah task force produced the EFM definition

Optical technologies point to point optical fiber @ 100Mbps 10 km

– Dual fiber duplex 100Base-LX10– Single fiber simplex 100Base-BX10

point to point optical fiber @ 1Gbps 10 km– Dual fiber duplex 1000Base-LX10– Single fiber simplex 1000Base-BX10

point to multipoint optical fiber @ 1Gbps 10/20 km (EPON )– Single fiber simplex 1000Base-PX10/20

Copper technologies point to point copper @ 10 Mbps 750 m (short reach PHY)

– VDSL 10PASS-TS

point to point copper @ 2 Mbps 2.7 km (long reach PHY)– SHDSL.bis 2Base-TL– up to 45 Mbps by bonding

OAM

Y(J)S EoS Slide 91

WAN-PHY WAN-PHY (10 GbE in STM-64)(10 GbE in STM-64)

There is a special case where Ethernet and SDH bit-rates are closeSTM-64 is 9953.28Mbps

GbE 10GBASE-R (64B/66B coding) can be directly mapped into a STM-64 (with contiguous concatenation) without need for GFP

MAC creates "stretched InterPacket Gap" to compensate for rate being < 10G

This is the fastest connection commonly used for Internet traffic

Complication: SDH clock accuracy is 4.6 ppm, GbE accuracy is 20 ppm

64*(270-9) = 16704 columns

J1

63 columns of fixed stuff

10GBASE-W 802.3-2005 Clause 50 G.707 Annex F

Y(J)S EoS Slide 92

Ethernet over WavelengthsEthernet over WavelengthsRather than muxing Ethernet flows using SONET mechanisms

We can allocate a separate wavelength (lambda) per flow

Wavelength Division Multiplexing (WDM)

For example, each wavelength may support OC-48 (2.5 Gbps)

Up to 8 channels is called coarse CWDM

More than 8 wavelengths (20 Gbps) is called dense DWDM

Present DWDM technology allows about 80 channels

Higher densities expected soon

DWDM’s tight channel spacing requires expensive cooled laser sources

Y(J)S EoS Slide 93

Ethernet PWsEthernet PWs

ProviderEdge

(PE)Customer

Edge

(CE)

CustomerEdge

(CE)

CustomerEdge

(CE)

Ethernet

MPLS network

PseudoWires (PWs)

CustomerEdge

(CE)

CustomerEdge

(CE)

ProviderEdge

(PE)Ethernet

Pseudowire (PW): Pseudowire (PW): mechanism that emulates essential mechanism that emulates essential attributes of a native service while transporting over a PSNattributes of a native service while transporting over a PSN

MPLS labelstack

PW label

PWEcontrolword

Ethernet frame(with or w/o FCS)