ittc high-performance networkingjpgs/courses/hsnets/lecture-switching-display.pdf ·...

226
High-Performance Networking The University of Kansas EECS 881 Switches and Routers © 2004–2008 James P.G. Sterbenz 1 October 2008 James P.G. Sterbenz Department of Electrical Engineering & Computer Science Information Technology & Telecommunications Research Center The University of Kansas [email protected] http://www.ittc.ku.edu/~jpgs/courses/hsnets rev. 1.3

Upload: vuongduong

Post on 09-Sep-2018

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

© James P.G. SterbenzITTCHigh-Performance Networking

The University of Kansas EECS 881Switches and Routers

© 2004–2008 James P.G. Sterbenz1 October 2008

James P.G. Sterbenz

Department of Electrical Engineering & Computer ScienceInformation Technology & Telecommunications Research Center

The University of Kansas

[email protected]

http://www.ittc.ku.edu/~jpgs/courses/hsnets

rev. 1.3

Page 2: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-2

© James P.G. SterbenzITTC

Switches and RoutersOutline

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 3: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-3

© James P.G. SterbenzITTC

Switches and RoutersOutline

SR.1. Switch overviewSR.2. Fast packet switchesSR.3. Switch fabrics

SR.4. Fast datagram switchesSR.5. Programmable, active,

and higher layer processing

network

application

session

transport

network

link

end system

networklink

node

network

link

nodenetwork

link

node

application

session

transport

network

link

end system

Page 4: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-4

© James P.G. SterbenzITTC

Network Node Principle S-II

Network nodes must support high-bandwidth low-latency, end-to-end flows, as well as their aggregation. High-speed network nodes should provide a scalable number or high-bandwidth, low delay interconnections.

Ideal NetworkNetwork Node Principle

networkCPU

M app

end system

CPU

M app

end systemD = 0

R = ∞

Page 5: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-5

© James P.G. SterbenzITTC

Switches and RoutersSR.1 Overview

SR.1 OverviewSR.1.1 Switch functionsSR.1.2 Traditional store-and-forward routersSR.1.3 LAN bridges and switchesSR.1.4 Ideal switch architecture

SR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 6: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-6

© James P.G. SterbenzITTC

Switches and RoutersSR.1.1 Switch Functions

SR.1 OverviewSR.1.1 Switch functionsSR.1.2 Traditional store-and-forward routersSR.1.3 LAN bridges and switchesSR.1.4 Ideal switch architecture

SR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 7: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-7

© James P.G. SterbenzITTC

Network LayerService and Interfaces

• Network layer 3 is above link layer 2– addressing : network layer identifier for end systems (hosts)– forwarding : transfers packets hop-by-hop

• using link layer services• network layer responsible for determining which next hop

– routing : determination of path to forward packets– signalling : messages to control network layer behaviour– traffic management : management of traffic and congestion

• Network layer service to transport layer (L4)– deliver TPDUs to destination transport entity

Page 8: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-8

© James P.G. SterbenzITTC

Network LayerForwarding vs. Routing

• Forwarding transfers packets at each hop– each switch (router) makes decision on which link to send– forwarding table (generally) used to make decision– forwarding is per packet decision

[analogy: determining which exits to take on a drive ]

• Routing determines the path to take– routing algorithm independent of forwarding– forwarding table entries populated by routing– routing is (generally) not done per packet

[analogy: planning trip from source to destination ]

Forwarding and routing are very different

Page 9: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-9

© James P.G. SterbenzITTC

Network LayerService and Interfaces

• Network layer packet may encapsulates link layer frame

transport layer

link layer

network layer

transport layer

link layer

network layer

TPDU TPDU

TPDUH TPDUH

Page 10: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-10

© James P.G. SterbenzITTC

Network Layer ServiceService Models: Best Effort

• Best effort– network attempts to deliver most packets most of the time– network may attempt to be fair among users– this is the Internet service model– contrast with best-effort applications

• Differentiated service• Guaranteed service

Page 11: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-11

© James P.G. SterbenzITTC

Network Layer ServiceService Models: Differentiated Service

• Best effort• Differentiated service

– some users or traffic types receive preferential treatment– may provide statistical targets for service performance– this model has been proposed as DiffServ for the Internet

• Guaranteed service

Page 12: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-12

© James P.G. SterbenzITTC

Network Layer ServiceService Models: Guaranteed Service

• Best effort• Differentiated Service• Guaranteed service

– some (perhaps statistical) guarantees on service• delivery (reliability)• performance (delay, bandwidth, jitter, etc)

– this model is provided by the PSTN and ATM networks– this model has been proposed by IntServ for the Internet

Page 13: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-13

© James P.G. SterbenzITTC

SwitchesFunctions: Overview

• Routing / signalling– per flow or longer

• Transfer control– per packet control

• Data manipulation– per byte or packet

routing and signalling

transfer control

routing algorithm

topology link state

traffic management

signalling

input processing switch fabric

output processing

management

data manipulation

link layer decapsulation

link layer framing

packet buffers

forwarding table

link scheduling

congestioncontrol

filter classify

fabric control

Page 14: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-14

© James P.G. SterbenzITTC

SwitchesFunctions: Routing and Signalling

• Routing / signalling– per flow or longer

• Management• Signalling• Topology database• Routing algorithm• Traffic

management

routing and signalling

transfer control

routing algorithm

topology link state

traffic management

signalling

input processing switch fabric

output processing

management

data manipulation

link layer decapsulation

link layer framing

packet buffers

forwarding table

link scheduling

congestioncontrol

filter classify

fabric control

Page 15: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-15

© James P.G. SterbenzITTC

SwitchesFunctions: Transfer Control

• Transfer control– per packet control

• Input control– classification tables– forwarding tables

• Congestion control• Fabric control

– e.g. set crosspoints

• Output control– link scheduling

routing and signalling

transfer control

routing algorithm

topology link state

traffic management

signalling

input processing switch fabric

output processing

management

data manipulation

link layer decapsulation

link layer framing

packet buffers

forwarding table

link scheduling

congestioncontrol

filter classify

fabric control

Page 16: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-16

© James P.G. SterbenzITTC

SwitchesFunctions: Data Manipulation

• Data manipulation– per byte or packet– performance critical

• Input processing– link layer– packet classification– output lookup

• Switch fabric• Packet buffers• Output processing

– packet scheduling– link layer

routing and signalling

transfer control

routing algorithm

topology link state

traffic management

signalling

input processing switch fabric

output processing

management

data manipulation

link layer decapsulation

link layer framing

packet buffers

forwarding table

link scheduling

congestioncontrol

filter classify

fabric control

Page 17: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-17

© James P.G. SterbenzITTC

Switches and RoutersSR.1.2 Store-and-Forward Routers

SR.1 OverviewSR.1.1 Switch functionsSR.1.2 Traditional store-and-forward routersSR.1.3 LAN bridges and switchesSR.1.4 Ideal switch architecture

SR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 18: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-18

© James P.G. SterbenzITTC

Store-and-Forward RoutersSecond Generation

• Second generation (1980s)– network interfaces (NI)

• share bus

– general purpose CPU

CPU M

network

w

NI

buffers

NI

link link

1

Page 19: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-19

© James P.G. SterbenzITTC

Store-and-Forward RoutersSecond Generation

• Second generation (1980s)– delays

• store-and-forward• contention for CPU• non-trivial header processing

CPU M

network

w

NI

buffers

NI

link link

2

Page 20: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-20

© James P.G. SterbenzITTC

Store-and-Forward RoutersSecond Generation

• Second generation (1980s)– delays

• store-and-forward• contention for CPU• non-trivial header processing

– buffering• in general purpose memory• contention for memory

CPU M

network

w

NI

buffers

NI

link link

3

Page 21: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-21

© James P.G. SterbenzITTC

Store-and-Forward RoutersSecond Generation

• Second generation (1980s)– delays

• store-and-forward• contention for CPU• non-trivial header processing

– buffering• in general purpose memory• contention for memory

– shared bus interconnect• packets traverse bus twice• severely limits # of ports• DMA transfers help

CPU M

network

w

NI

buffers

NI

link link

4

Page 22: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-22

© James P.G. SterbenzITTC

Store-and-Forward RoutersThird Generation

• Third generation (1990s)– NI (network interface)

• NP: embedded controller– e.g. i960

• packet processing• packet buffers

CPU M

w

NI

buffers

NP

NI

buffers

NP

network

L2 L2

L3L3

1

Page 23: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-23

© James P.G. SterbenzITTC

Store-and-Forward RoutersThird Generation

• Third generation (1990s)– NI (network interface)

• NP: embedded controller– e.g. i960

• packet processing• packet buffers

– third party bus transfer• bus is switch fabric• single transfer per packet

– still significant bottleneck

CPU M

w

NI

buffers

NP

NI

buffers

NP

network

L2 L2

L3L3

2

Page 24: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-24

© James P.G. SterbenzITTC

Switches and RoutersSR.1.3 LAN Bridges and Switches

SR.1 Switch overviewSR.1.1 Switch functionsSR.1.2 Traditional store-and-forward routersSR.1.3 LAN bridges and switchesSR.1.4 Ideal switch architecture

SR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 25: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-25

© James P.G. SterbenzITTC

Switches and RoutersLAN Switching

• Recall LAN interconnection technology progression:– single segment– learning bridge– hub– switch

Page 26: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-26

© James P.G. SterbenzITTC

LAN SwitchingExact Match Lookup

• Most LANs use large flat address space– IEEE 802 48-bit MAC address

Challenges in switching?

Page 27: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-27

© James P.G. SterbenzITTC

LAN SwitchingExact Match Lookup

• Most LANs use large flat address space– IEEE 802 48-bit MAC address

• No structured aggregation of addresses– 248 ≈ 2.8×1015 entry address space– requires exact match lookup

• Limits size of flat-address networks

Page 28: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-28

© James P.G. SterbenzITTC

LAN SwitchingExact Match Lookup Techniques

• Binary search– O (log n) time steps– requires sorted entries

• Hashing– O (1) time steps best case– O (n) time steps worst case due to hash collisions

• Perfect hashing– upper bound on worst case lookup time– hash function modified as address entries added to table

Page 29: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-29

© James P.G. SterbenzITTC

Switches and RoutersSR.1.4 Ideal Switch Architecture

SR.1 Switch overviewSR.1.1 Switch functionsSR.1.2 Traditional store-and-forward routersSR.1.3 LAN bridges and switchesSR.1.4 Ideal switch architecture

SR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 30: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-30

© James P.G. SterbenzITTC

Ideal Switch ArchitectureBandwidth, Latency, Ports

• Infinite bandwidth• Zero latency• Unlimited

number of ports

Store-and-Forward & Queueing Delay Minimisation S-II.3

Store-and-forward delays should be avoided, and per packet queueing should be minimised. In the ideal case, nodes should pipeline and cut through packet with zero per packet delays.

n = ∞

R = ∞ D = 0

Page 31: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-31

© James P.G. SterbenzITTC

Switches and RoutersSR.2 Fast Packet Switches

SR.1 Switches and routersSR.2 Fast packet switches

SR.2.1 Input and label processingSR.2.2 Packet structureSR.2.3 Functional partitioning

SR.4 Switch fabric architectureSR.5 Fast datagram switchesSR.6 Programmable, active, & higher layer processing

Page 32: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-32

© James P.G. SterbenzITTC

Fast Packet SwitchMotivation

• Connection state– simple per packet

processing

• Switch fabric– eliminate blocking– no store-and-forward

Fast Packet Switch Critical Path Principle S-1bc

Simplify and optimise per byte and per packet data manipulation and transfer control functions for implementation in a hardware critical path. State maintained per connection streamlines per packet processing.

input processing output processing

routing and signalling

switch fabric

link scheduling

link

link

link

link

label swap

CID table

switch fabric control

Page 33: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-33

© James P.G. SterbenzITTC

Fast Packet SwitchArchitecture

• Routing & signalling– populates forwarding

• connection id table

• Input processing– decodes headers– connection id lookup

• determines output

• Switch fabric– control sets paths to interconnects inputs to outputs

• Output processing– buffer for contention and schedule for transmission

input processing output processing

routing and signalling

switch fabric

link scheduling

link

link

link

link

label swap

CID table

switch fabric control

Page 34: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-34

© James P.G. SterbenzITTC

Fast Packet SwitchArchitecture

• Input processing– connection id table with label swapping– switch fabric– other input functions Lecture IO

• Switch fabric• Output processing Lecture IO• Routing and signalling EECS780 lecture NR

Page 35: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-35

© James P.G. SterbenzITTC

Switches and RoutersSR.2.1 Input and Label Processing

SR.1 Switches and routersSR.2 Fast packet switches

SR.2.1 Input and label processingSR.2.2 Packet structureSR.2.3 Functional partitioning

SR.4 Switch fabric architectureSR.5 Fast datagram switchesSR.6 Programmable, active, & higher layer processing

Page 36: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-36

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping

• Connection id is index into table

cin pout cout cstate

=

payload payload cin

1

Page 37: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-37

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping

• Connection id is index into table• Table entry gives egress link and next hop id

cin pout cout cstate

=

payload cout payload cin

2

Page 38: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-38

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping

• Connection id is index into table• Table entry gives egress link and next hop id

– port may be prepended for self-routing fabricsexamples: ATM, MPLS

cin pout cout cstate

=

pout payload cout payload cin

3

Page 39: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-39

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch

1

SETUP (as, ad, r)

Page 40: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-40

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing

• resolve dest. address ad to output port pout from topo. database

2

pout

SETUP (as, ad, r)

Page 41: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-41

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing

• resolve dest. address ad to output port pout from topo. database

– check for available resources r in switch and egress link• if available temporarily reserve

3

pout

SETUP (as, ad, r)

Page 42: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-42

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing

• resolve dest. address ad to output port pout from topo. database

– check for available resources r in switch and egress link• if available temporarily reserve

– choose unused connection id cout

• temporarily put output port pout in connection forwarding table

4

c:p

Page 43: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-43

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing

• resolve dest. address ad to output port pout from topo. database

– check for available resources r in switch and egress link• if available temporarily reserve

– choose unused connection id cout

• temporarily put output port pout in connection forwarding table

– pass SETUP to next hop• save SETUP association (as, ad, id) to match with CONNECT

5

c:p

SETUP

Page 44: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-44

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing

• resolve dest. address ad to output port pout from topo. database

– check for available resources r in switch and egress link• if available temporarily reserve

– choose unused connection id cout

• temporarily put output port pout in connection forwarding table

– pass SETUP to next hop• save SETUP association (as, ad, id) to match with CONNECT

– release all state if:• corresponding RELEASE message received• corresponding CONNECT not received within timeout interval 6

c:p

SETUP

Page 45: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-45

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing– check for and temporarily reserve resources– choose and insert unused connection id– pass SETUP to next hop

• CONNECT message returned to switch

7

c:p

CONNECT

Page 46: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-46

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing– check for and temporarily reserve resources– choose and insert unused connection id– pass SETUP to next hop

• CONNECT message returned to switch– commit switch and egress link resources

8

c:p

CONNECT

Page 47: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-47

© James P.G. SterbenzITTC

Fast Packet SwitchConnection State Establishment

• SETUP message arrives at switch– connection routing– check for and temporarily reserve resources– choose and insert unused connection id– pass SETUP to next hop

• CONNECT message returned to switch– commit switch and egress link resources– pass CONNECT message back through ingress link

9

c:pCONNECT

Page 48: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-48

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p0

3 1

5 1

p3

p2

p1

3 3

1 4

1 6

0

pout cout

7 1

1

Page 49: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-49

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p0

3 1

5 1

p3

p2

p1

3 3

1 4

1 6

0

pout cout

7 1

2

Page 50: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-50

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p0

3 1

3

5 1

p3

p2

p1

3 3

1 4

1 6

0

pout cout

7 1

3

Page 51: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-51

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p0

7 2

2

3 1

3

5 1

p3

p2

p1

3 3

1

21

4 0

1 6

3

25

0

0

pout cout

7 1

0 2

4

Page 52: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-52

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p2

p0

7 2

2

3 1

3

5 1

p3

p2

p1

3 3

1

21

4 0

1 6

3

25

0

0

pout cout

7 1

0 2

5

Page 53: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-53

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p2

p0

7 2

2

3 1

37

5 1

p3

p2

p1

3 3

1

21

4 0

1 6

3

25

0

0

pout cout

7 1

0 2

6

Page 54: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-54

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p2

p0

3

7 2

2

3 1

5 3

37

5 1

p3

p2

p1

3 3

1

233

1

4 0

0

1 6

3

2

25

0

0 0

pout cout

7 21

0 2

3

7

Page 55: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-55

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p3 p2

p0

3

7 2

2

3 1

5 3

37

5 1

p3

p2

p1

3 3

1

233

1

4 0

0

1 6

3

2

25

0

0 0

pout cout

7 21

0 2

3

8

Page 56: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-56

© James P.G. SterbenzITTC

Fast Packet SwitchLabel Swapping Example

p3 p2

p0

3

7 2

2

3 1

5 3

37

5

5 1

p3

p2

p1

3 3

1

233

1

4 0

0

1 6

3

2

25

0

0 0

pout cout

7 21

0 2

3

9

Page 57: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-57

© James P.G. SterbenzITTC

Fast Packet SwitchConnectionless vs. Connection Tradeoff

• Connection-oriented fast packet switching– requires round trip connection setup latency– achieved higher data rate due to simple label swap

• IP lookup was a bottleneck in 1980s

Connectionless vs. Connection Tradeoff S-5A

The latency of connection setup must be traded against the increase in packet processing rate and resultant bandwidth increase achievable due to the less complex label swapping.

Page 58: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-58

© James P.G. SterbenzITTC

Switches and RoutersSR.2.2 Packet Structure

SR.1 Switches and routersSR.2 Fast packet switches

SR.2.1 Input and label processingSR.2.2 Packet structureSR.2.3 Functional partitioning

SR.4 Switch fabric architectureSR.5 Fast datagram switchesSR.6 Programmable, active, & higher layer processing

Page 59: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-59

© James P.G. SterbenzITTC

Packet StructureSize and Variability

• PDU size and structure are critical to performance

Packet Size and Variability Tradeoff S-8A

Packet size is a balance of a number of parameters that affect performance. Trade the statistical multiplexing and fine-grained control benefits of small packets against the efficiency of large packets. Trade the benefits to switch design of fixed cell sizeagainst the overhead of SAR and efficiency of variable packet size. Multiple discrete packet sizes increase flexibility at the cost of additional switch complexity. Hop-by-hop SAR localizes packet size optimizations at the cost of additional per hop delay and complexity.

Page 60: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-60

© James P.G. SterbenzITTC

Packet Size and StructureVariability

• Fixed vs. variable size packetstradeoffs?

Page 61: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-61

© James P.G. SterbenzITTC

Packet Size and StructureVariability

• Fixed size (cells)+ easier to design switches– difficult to predetermine the best size

• Variable size– more difficult to design switches+ no need for agreement on size+ less need for fragmentation/segmentation

• Discrete sizes: advantages of both fixed and variable• integral multiples, e.g. 64B, 128B, 192B…• power-of-2 scaling with data rate

e.g. 128B @ OC-3, 256B @ OC-12, 512B @ OC-48

Page 62: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-62

© James P.G. SterbenzITTC

Packet Size and StructureSize

• Small vs. large packetstradeoffs?

Page 63: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-63

© James P.G. SterbenzITTC

Packet Size and StructureSize

• Small packets+ efficient statistical multiplexing– high header/payload overhead– short interarrival time challenge per packet processing

• note: this is one major reason ATM failed

• Large packets+ significantly easier per packet processing– less efficient statistical multiplexing– larger queueing delays+ efficient transport of large data blocks– inefficient transport of signalling and control messages

e.g. TCP ACKs

Page 64: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-64

© James P.G. SterbenzITTC

Packet Size and StructureGranularity

• Important to match granularity to packet processingwhy?

Page 65: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-65

© James P.G. SterbenzITTC

Packet Size and StructureGranularity

• Important to match granularity to packet processing+ byte / octet (8 bits)

• control fields should align to 8-bit boundaries

+ word (typically 32 bits)• most end-system processing at word granularity• payload should align to 32-bit boundaries

+ end system data unit• system buffers and memory structures• power-of-2 size will likely be integral fraction

+ commodity memory components• power-of-2 size

– note: ATM cell was none of these

Page 66: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-66

© James P.G. SterbenzITTC

Packet Size and StructurePacket Format

• Header– fields that determine

packet processing

• Payload– TPDU transport protocol data unit

• Trailer– fields that are dependent on

packet processing• e.g. checksum to allow cut-through

TPDU

check

header

trailer

payload

payload lengthcheck

typeconnection id

version

QoSauthentication

Page 67: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-67

© James P.G. SterbenzITTC

Packet Size and StructurePacket Format: Header

• Headerfields that determinepacket processing– protocol version– packet type: control, data, etc.– connection id or protocol demux– QoS and authentication if needed– header check

• processing even if data corrupted

• Payload• Trailer

TPDU

check

header

trailer

payload

payload lengthcheck

typeconnection id

version

QoSauthentication

Page 68: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-68

© James P.G. SterbenzITTC

Packet Size and StructurePacket Format: Payload

• Header• Payload

– TPDU transport protocol data unit

• Trailer

TPDU

check

header

trailer

payload

payload lengthcheck

typeconnection id

version

QoSauthentication

Page 69: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-69

© James P.G. SterbenzITTC

Packet Size and StructurePacket Format: Trailer

• Header• Payload• Trailer

fields that are dependent onpacket processing– checksum to allow cut-through

• compute and compare TPDU

check

header

trailer

payload

payload lengthcheck

typeconnection id

version

QoSauthentication

Page 70: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-70

© James P.G. SterbenzITTC

Packet Size and StructureControl Fields

• Control field structure and encoding is critical– simple encoding (bit vectors vs. code points)– byte/octet granularity and alignment– field length

• fixed when possible• variable length prepended with length (skip vs. hunt)

Packet Control Field Structure S-8B

Optimise packet header and trailer fields for efficient processing. Fields should be simply encoded, byte aligned, and fixed length where possible. Variable-length fields should be prepended with their length.

Page 71: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-71

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM Cells

• ATM cell format– fast packet switching– fine-grained statistical multiplexing

• Size determined by ITU committee compromise– 48B = avg(32, 64)

• 64 from US = min of proposals for data (and voice)• 32 from European PTTs to avoid voice echo cancellers

• Problems:– header tiny to keep overhead low; no room for seq #– nothing a power of 2– 48B + 5B = 53B; not even a multiple of 8

Page 72: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-72

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM AAL-5 Cells

PTI 0X1

VPI

VCI CLP

HEC

CRC-32 length CPI

CPCS-UU

SAR-SDU ≤ 40B

last cell – SDU type 1 other cells – SDU type 0

PTI 0X0

VPI

VCI CLP

HEC

SAR-SDU = 48B

header5B

payload

trailer

48B

40B

8B

• AAL 3/4 for data– 4B AAL header

• useless seq#

– 44B payload• not div 8

• AAL 5– no header– trailer last frag– forced on ITU

Page 73: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-73

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM Hierarchical Switching

• ATM split connection id– permits 2-level switching hierarchy

• VPI (virtual path identifier): high order bits– switched by core and edge switches– core switches have smaller VP tables– switch large bundles of VCs in each VP with single lookup

• VCI (virtual channel identifier): low order bits– switched by local and edge switches– local and edge switches handle small subset of overall nodes

Page 74: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-74

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM Hierarchical Switching

• VC LAN switch– switches small number of VCs

VC

VC

Page 75: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-75

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM Hierarchical Switching

• VC LAN switch– switches small number of VCs

• VPVC edge switch– bundles large numbers of VCs into VPs and switches

VPVC

VCVC

VC

Page 76: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-76

© James P.G. SterbenzITTC

Packet StructureExample5.4 ATM Hierarchical Switching

• VC LAN switch– switches small number of VCs

• VPVC edge switch– bundles large numbers of VCs into VPs and switches

• VP core switch– switches VPs; far fewer than VPVCs

VP

VPVC

VCVC

VCVP

Page 77: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-77

© James P.G. SterbenzITTC

Packet Size and StructureSegmentation and Reassembly

• End-to-end+ more efficient

in the network– requires global

agreement on packet size• original assumption for ATM

• cell size standardised• SAR difficult in 1980’s

SAR SAR

SAR SAR

Page 78: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-78

© James P.G. SterbenzITTC

Packet Size and StructureSegmentation and Reassembly

• End-to-end+ more efficient

in the network– requires global

agreement on packet size

• Hop-by-hop– significant overhead

per switch+ each switch can use cell size

optimised to its architecture• used by modern

cell-based IP routers

SAR SAR

SAR

SAR

SAR SAR

SAR

SAR

Page 79: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-79

© James P.G. SterbenzITTC

Packet StructureExample5.5 MPLS Shim

network layer packet

stacked labels single label

S=1

label

COS TTL

link layer header

link layer header

label4B

shim

labelstack

link layer trailer link layer trailer

network layer packet

S=1

label

COS TTL

S=0

label

COS TTL

S=0

label

COS TTL

• MPLS multiprotocol label switching– fast packet switching for IP

• Add label shim– switches swap label– stacked labels

• allows net hierarchy(ala VP/VC)

Page 80: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-80

© James P.G. SterbenzITTC

Traffic ManagementConnection-Oriented Fast Packet Switching

• Traffic management– required to protect high-performance flow characteristics– functions in critical path must be implemented at line speed

• simple hardware required for third generation (1990s)

Switch Traffic Management S-II.2

In a resource-constrained environment, switches must support admission control with resource reservations to provide guaranteed service traffic classes and policing to prevent othertraffic form interfering.

Page 81: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-81

© James P.G. SterbenzITTC

Traffic ManagementOverengineering vs. Optimality

• Traffic management can be very complex• Overengineering can dramatically simplify

Overengineering vs. Optimality S-2B

Optimal traffic management policies use processing and memory, which also adds latency to connection establishment; trade these against the bandwidth wasted by overengineering and overprovisioning the network.

Page 82: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-82

© James P.G. SterbenzITTC

Switches and RoutersSR.2.3 Functional Partitioning

SR.1 OverviewSR.2 Fast packet switches

SR.2.1 Input and label processingSR.2.2 Packet structureSR.2.3 Functional partitioning

SR.4 Switch fabric architectureSR.5 Fast datagram switchesSR.6 Programmable, active, & higher layer processing

Page 83: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-83

© James P.G. SterbenzITTC

Switch Functional PartitioningDesign and Technology Alternatives

• Hardware vs. Software– control CPU vs. embedded controller vs. network processor

• Custom VLSI vs. ASIC vs. FPGA• Technology

– DRAM vs. SRAM vs. TCAM– CMOS vs. GaAs

• Electronic vs. optical

Functional Partitioning and Assignment Principle S-1C

Carefully determine what switch functionality is to be implemented in scarce or expensive technology

Page 84: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-84

© James P.G. SterbenzITTC

Switch Functional PartitioningElectronic vs. Optical

• Optical switches– avoid O/E/O conversion (optical/electronic/optical)

• Data path– optical switch elements in photonic domain

• Control pathproblem?

Page 85: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-85

© James P.G. SterbenzITTC

Switch Functional PartitioningElectronic vs. Optical

• Optical switches– avoid O/E/O conversion (optical/electronic/optical)

• Data path– optical switch elements in photonic domain

• Control path: photonic processing infeasible– not possible to process header optical domain– conventional buffering not possible in optical domain

solution?

Page 86: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-86

© James P.G. SterbenzITTC

Switch Functional PartitioningElectronic vs. Optical

• Optical switches– avoid O/E/O conversion (optical/electronic/optical)

• Data path– optical switch elements in photonic domain

• Control path: photonic processing infeasible– O/E conversion on header only

• header processed by conventional digital electronics

– optical delay lines (long tiny fiber loops)• delay photonic payload long enough for:

– header processing– relatively slow optical switch element control

Page 87: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-87

© James P.G. SterbenzITTC

Switch Functional PartitioningElectronic vs. Optical

• Photonic processing infeasible

• Split header O/E/O– electronic

control– optical packet

delay

input processing

routing and signalling

optical switch fabric

electronic switch fabric control

input processor

o/e

forward table

e/o

input processor

o/e

forward table

e/o

Page 88: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-88

© James P.G. SterbenzITTC

Switches and RoutersSR.3 Switch Fabric Architecture

SR.1 Switches and routersSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 89: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-89

© James P.G. SterbenzITTC

Fast Packet SwitchSwitch Fabric Architecture

• Connection state– simple per packet

processing

• Switch fabric– eliminate blocking– no store-and-forward

input processing output processing

routing and signalling

switch fabric

link scheduling

link

link

link

link

label swap

CID table

switch fabric control

Page 90: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-90

© James P.G. SterbenzITTC

Switches and RoutersSR.3.1 Queueing Analysis Background

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 91: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-91

© James P.G. SterbenzITTC

Queueing Analysis BackgroundBasic Model

• Packets arrive in a queue at rate λ [1/s]• Packets are served at a rate μ [1/s]

– service time of s [s]

• System utilisation is ρ [erlangs] ([1] = [s/s])– expressed as number in (0…1) or %

λ arrival rateρ utilisation

queue server μ

Page 92: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-92

© James P.G. SterbenzITTC

Queueing Analysis BackgroundLittle’s Theorem

• N = λT• Variables

N – average number of customers (packets) in systemλ – arrival rate [1/s]T – average time spent in system [s]

• Holds for all distributions• Intuition

– as service time increases, so do number in system– as arrival rate increases, so do number in system

Page 93: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-93

© James P.G. SterbenzITTC

Queueing Analysis BackgroundDistributions

• Arrival and service rates determine performance– average rates matter, but…

• Distribution of rates are critical– uniform: all interarrival or service times equal

• simple but unrealistic for most systems

– exponential or Poisson (independent: memoryless)• Pr[k arrivals in interval T] =

– bursty: arrivals clumped into groups– self-similar: distribution similar at different granularities

• evidence of self-similar traffic in Internet• challenges benefits of statistical multiplexing

( )!

kTT

ek

λλ −

Page 94: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-94

© James P.G. SterbenzITTC

Queueing Analysis BackgroundNotation

• A/S/N– A and S: distribution of arrivals and service times

G – general independentM – exponential (Poisson)

• memoryless: each event independent of the previous

D – deterministic or uniform– N: number of servers

• Examples– M/M/1: exponential arrival, exponential service, single server

Page 95: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-95

© James P.G. SterbenzITTC

M/M/1 Queueing AnalysisBasic Results

• Utilization ρ = λ / μ– arrival rate / service rate

• Average time in queue– note what happens as

ρ → 1

• Average time in system

/1

W ρ μρ

=−

1/1

T μρ

=−

ρ 1.0

W

Page 96: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-96

© James P.G. SterbenzITTC

M/M/1 Queueing AnalysisDiscussion

• M/M/1 equations very simple– so they are frequently used in closed-form analysis– reflects independent arrivals

• Web transactions (HTTP gets)• voice call SETUPs

– does not accurately represent traffic in network• bursty traffic (e.g. P2P file sharing, TCP segment bursts)• correlated traffic

• G/G/1 equations very complex!– requires integration of a function– difficult to analyse networks of generalised queues

Page 97: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-97

© James P.G. SterbenzITTC

Analysis TechniquesAnalytical

• Analytical queueing analysis– mathematical analysis of system– useful for small simple system– exact behaviour of complex system difficult or impossible

• not possible to precisely analyse large network of real traffic

– critical problem:• simplifying assumptions must be made for complex systems• usefulness of results depends on these assumptions!

• Simulation• Emulation• Measurement

Page 98: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-98

© James P.G. SterbenzITTC

Analysis TechniquesSimulation

• Queueing analysis• Simulation (e.g. ns-3)

– build software simulation model of system• generally more complex models possible than analytical

– critical problem:• real system must be abstracted to software model• abstractions usually contain simplifying assumptions• usefulness of results depends on abstraction and model!

• Emulation• Measurement

Page 99: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-99

© James P.G. SterbenzITTC

Analysis TechniquesEmulation

• Queueing analysis• Simulation• Emulation

– hardware component designed to behave like real system• e.g. link emulator

– allows control of behaviour not possible in real system– critical problem

• emulator usually simplification of real system• usefulness of results depends on simplification!

– emulator frequently a component of simulation or testbed

• Measurement

Page 100: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-100

© James P.G. SterbenzITTC

Analysis TechniquesMeasurement

• Queueing analysis• Simulation• Emulation• Measurement

– measurement of actual system• may be a testbed – usually a small replica of real network

– critical problem:• testbed may not reflect scale of real network• access to measurement data difficult

– due to scale of real network– due to administrative concerns, including privacy

Page 101: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-101

© James P.G. SterbenzITTC

Analysis TechniquesVerification

• Combination of techniques– each technique has strengths and weaknesses– should be combined as necessary

• Verification of models– compare results of two techniques to verify model– example:

• compare queueing and simulation model of simple system• gives confidence of simulation model for more complex system

Page 102: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-102

© James P.G. SterbenzITTC

Analysis TechniquesDesign Cycle

• Use each technique as appropriate in design cycle• Design cycle

– mathematical analysis and/or simulation• iterate on design parameters until desired behaviour• gives insight on how to build prototype system

– construct prototype and measure in testbed• analysis in small scale• iterate design; modify simulation model if necessary

– build real system on large scale• measure to confirm design and modify future deployments

Page 103: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-103

© James P.G. SterbenzITTC

Switches and RoutersSR.3.2 Buffering

SR.1 Switches and routersSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 104: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-104

© James P.G. SterbenzITTC

Switch Fabric ArchitectureOverview

• Switch fabric– core of switch– connects arbitrary

inputs to outputs

• Goals– eliminate blocking– no store-and-forward

input processing output processing

routing and signalling

switch fabric

link scheduling

link

link

link

link

label swap

CID table

switch fabric control

Page 105: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-105

© James P.G. SterbenzITTC

Switch Fabric ArchitectureBlocking

• Blocking characteristics (among different outputs)– strictly nonblocking: under all conditions– wide-sense nonblocking: if particular algorithm is used– rearrangeably nonblocking: if existing paths are rearranged– virtually nonblocking: with extremely low probability

Nonblocking Switch Fabric Principle S-II.4f

A nonblocking switch fabric is the core or a high performance switch. Avoid blocking by space-division parallelism, internal speedup, and internal pipelined buffering with cut-through. Strict and wide-sense nonblocking avoids the complexity and delay of path rearrangement.

Page 106: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-106

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Contention (burst collisions) in a non-blocking fabric– occurs when traffic destined for same output– requires buffering even for well-behaved traffic

collisions

in1

in2

out delayed

Page 107: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-107

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Unbuffered fabric

• Internal buffers

Page 108: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-108

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueingproblem?

Page 109: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-109

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

0

Page 110: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-110

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

1

Page 111: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-111

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

2

Page 112: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-112

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

3

Page 113: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-113

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

4

Page 114: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-114

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

5

Page 115: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-115

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

6

Page 116: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-116

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

• Output queueingproblem?

Page 117: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-117

© James P.G. SterbenzITTC

Switch Fabric ArchitectureContention and Buffering

• Input queueing– suffers from

head-of-line blocking

• Output queueing– requires either:

• internal speedup• internal expansion

Page 118: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-118

© James P.G. SterbenzITTC

Switch Fabric ArchitectureHead-of-Line Blocking

Head-of-Line Blocking Avoidance Principle S-II.4q

Avoid head-of-line blocking. Output queueing requires internal speedup, expansion, or buffering. Virtual output queueing requires additional queues or queueing complexity. The two techniques must be traded against one another, and can be used in combination.

Page 119: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-119

© James P.G. SterbenzITTC

Switch Fabric ArchitectureVirtual Output Queueing

• Virtual output queueing– parallel buffers– non-FIFO buffers

Page 120: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-120

© James P.G. SterbenzITTC

Switches and RoutersSR.3.3 Single-Stage Elements

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 121: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-121

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Bus as a Switch

• Simple design shared medium bus– point of contention: only one input active at a time

• 2nd/3rd generation routers

– suitable for small switches

• Multicast– inherent broadcast

i0 i1 i2 i3 i4 i5 i6 i7

o0 o1 o2 o3 o4 o5 o6 o7

w

Page 122: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-122

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Shared Memory Switch

• Simple design– packets written by input– packets read by output

• Shared memory– point of contention– speedup necessary

• but access times not scaling with Moore’s

• Multicast– multiple writes or– multicast output demux

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7

output demultiplex

i nput

mul t i pl ex

shared memory

Page 123: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-123

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

0

straight cross

Page 124: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-124

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

1

Page 125: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-125

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

2

Page 126: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-126

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

3

straight cross

Page 127: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-127

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

1

Page 128: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-128

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

2

Page 129: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-129

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

3

Page 130: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-130

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Basic 2×2 Switch Element

• States– point-to-point

• straight• cross

– multicast• Types

– internally buffered or unbuffered

– self routing orexternally controlled

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross

Page 131: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-131

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Crossbar Switch

• Crosspoint switch element– electronic

– optical MEMS• rotating mirror

column

ii

oj

electronic optical MEMS

ii

oj

column

cross turn

Page 132: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-132

© James P.G. SterbenzITTC

Switch Fabric ArchitectureSingle Stage: Crossbar Switch

• Crossbar fabric– square array of

crosspoint elements– O (n 2) growth

complexity– reasonable for

moderate n

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7

Page 133: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-133

© James P.G. SterbenzITTC

Crossbar SwitchPath Selection

• Crossbar fabric– simple path routing

• element (o,i) turns

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 0

Page 134: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-134

© James P.G. SterbenzITTC

Crossbar SwitchPath Selection

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 1

Page 135: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-135

© James P.G. SterbenzITTC

Crossbar SwitchPath Selection

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 2

Page 136: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-136

© James P.G. SterbenzITTC

Crossbar SwitchPath Selection

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 3

Page 137: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-137

© James P.G. SterbenzITTC

Crossbar SwitchStrictly Nonblocking

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• ij→on noblock ik→om∀j,k,n,m: i ≠ j, n ≠ m

• i1 → o1

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 1

Page 138: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-138

© James P.G. SterbenzITTC

Crossbar SwitchStrictly Nonblocking

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• ij→on noblock ik→om∀j,k,n,m: i ≠ j, n ≠ m

• i1 → o1

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 2

Page 139: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-139

© James P.G. SterbenzITTC

Crossbar SwitchStrictly Nonblocking

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• ij→on noblock ik→om∀j,k,n,m: i ≠ j, n ≠ m

• i1 → o1

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 3

Page 140: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-140

© James P.G. SterbenzITTC

Switches and RoutersSR.3.4 Multistage Switches

SR.1 Switches and routersSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 141: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-141

© James P.G. SterbenzITTC

Switch FabricsMultistage Switches

• Large switches built from single stage elements– 2×2 elements or n ×n crossbars– O (n log n) growth complexity

Switch Scalability S-III.4

The construction of switches with a large port count requires scalable switch fabrics. Exploit regular structures for implementation in VLSI or optics as the basis for recursively constructed fabrics with logarithmic growth complexity.

Page 142: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-142

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction and Scalability

• Delta fabric– recursive

construction– double fabric

vertically– add shuffle

stage at input

0

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Page 143: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-143

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction and Scalability

• Delta fabric– O (n log n )

n /2 rows log2 n stagesn = 22/2 log2 2 = 1

i0 i1

o0 o1

s0

1

Page 144: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-144

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction and Scalability

• Delta fabric– O (n log n )

n /2 rows log2 n stagesn = 44/2 log2 4 = 4

i0

i2

i1

i3

o0

o2

o1

o3

s1 s0

2

Page 145: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-145

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction and Scalability

• Delta fabric– O (n log n )

n /2 rows log2 n stagesn = 88/2 log2 8 = 12

i0

i2

i4

i6

i1

i3

i5

i7

o0

o2

o4

o6

o1

o3

o5

o7

s2 s1 s0

3

Page 146: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-146

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction and Scalability

• Delta fabric– O (n log n )

n /2 rows log2 n stagesn = 1616/2 log216=32

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

4

Page 147: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-147

© James P.G. SterbenzITTC

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

0

Page 148: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-148

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

1

Page 149: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-149

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

1010

1010

2

Page 150: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-150

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

31010

Page 151: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-151

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

41010 1010

Page 152: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-152

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

51010 1010

Page 153: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-153

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

61010 1010 1010

Page 154: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-154

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

71010 1010 1010

Page 155: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-155

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

81010 1010 1010 1010

Page 156: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-156

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10 1010

91010 1010 1010 1010

Page 157: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-157

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

1010 1010 1010 1010

1010

10

Page 158: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-158

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

11

Page 159: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-159

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

12

Page 160: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-160

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

131010

Page 161: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-161

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

141010

Page 162: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-162

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

151010 1010

Page 163: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-163

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

161010 1010

Page 164: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-164

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

171010 1010 1010

Page 165: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-165

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010

1010

181010 1010 1010

Page 166: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-166

© James P.G. SterbenzITTC

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o15

s3 s2 s1 s0

Multistage Switch FabricsDelta Fabric Construction Self-Routing

• Delta fabric– self-routing– ith bit of pout

used to make routing decision in ith stage

– i2 → o10

– i13 → o10

1010 1010 1010 1010

1010

19

Page 167: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-167

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Overview

• Clos fabric– n×n fabric is interconnection of 3 crossbars stages– input stage of n/d elements:

d×r crossbars each with d :r expansion– middle stage of r elements: n/d×n/d square crossbars– output stage of n/d elements:

r×d crossbars each with r :d concentration– perfect shuffle interconnection between stages:

i th output of j th crossbar → j th input of i th crossbar

Page 168: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-168

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

0

d

n

rd×rd<r

0

n/d –1

r×dr>d

d

n

0

n/d –1

r–1

10

n/d×n/d

r × n/d

Page 169: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-169

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

1

Page 170: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-170

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

2

Page 171: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-171

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

3

Page 172: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-172

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

4

Page 173: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-173

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

5

Page 174: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-174

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

6

Page 175: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-175

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

7

Page 176: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-176

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

8

Page 177: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-177

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

9

Page 178: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-178

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

10

Page 179: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-179

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

11

Page 180: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-180

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

12

Page 181: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-181

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

13

Page 182: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-182

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

14

Page 183: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-183

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Architecture and Construction

15

Page 184: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-184

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing and Blocking

• Clos fabric path routing– for input from i th crossbar in first stage →

j th crossbar in last stage– choose any middle stage crossbar with (i, j) point available

• Blocking characteristics– blocking depends on expansion in middle stages r– strictly nonblocking for unicast iff r ≥ 2d –1– engineered to balance cost against blocking probability

Page 185: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-185

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing

0

Page 186: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-186

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing

1

Page 187: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-187

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing

2

Page 188: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-188

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing

3

Page 189: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-189

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Routing

4

Page 190: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-190

© James P.G. SterbenzITTC

Multistage Switch FabricsClos Fabric Application

• Clos fabric– used in modern switch fabrics– e.g. Ciena optical switches, Juniper routers

• Engineering tradeoff optimisations– crossbar elements to VLSI switch complexity– partitioning: 1st and last stages can be on line-group cards

• permits switching among different rate interfaces

Page 191: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-191

© James P.G. SterbenzITTC

Multistage Switch FabricsExample5.C Ciena CoreDirector Optical Switch

• Ciena CoreDirector optical switch– carrier-grade switch (WAN/MAN service provider market)– note: used in the KU/KSU/UMKC/UNL GpENI project

• Backplane– 640 Gb/s aggregate capacity– center stage of Clos fabric

• Interface slots– 1st and 3rd stages of Clos fabric– SONET OC-3 – OC-192, OTN OUT-2, 1 – 10 GbEthernet

Page 192: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-192

© James P.G. SterbenzITTC

Switches and RoutersSR.3.5 Multicast Support

SR.1 Switches and routersSR.2 Fast packet switchesSR.3 Switch fabric architecture

SR.3.1 Queueing analysis backgroundSR.3.2 BufferingSR.3.3 Single-stage elementsSR.3.4 Multistage switchesSR.3.5 Multicast support

SR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

Page 193: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-193

© James P.G. SterbenzITTC

Switch Fabric ArchitectureNative Multicast

• Multicast support in network conserves bandwidthWhy?

Page 194: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-194

© James P.G. SterbenzITTC

Switch Fabric ArchitectureNative Multicast

• Multicast support in network conserves bandwidth• Multicast support in switches

– prevents multiple transmissions of a single multicast packet– reduces delay in subsequent packets

Switches Should Provide Native Multicast S-2C

Switches should provide native multicast support to conserve bandwidth and avoid latency of repeated transmission.

Page 195: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-195

© James P.G. SterbenzITTC

Crossbar Switch MulticastCrosspoint Element Duplication

• Crosspoint switch element– electronic

• multicast possible

column

ii

oj

electronic

cross turn duplicate

Page 196: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-196

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• i1 → o1

– multicast• i1 → {o1, o6 }

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 1

Page 197: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-197

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• i1 → o1

– multicast• i1 → {o1, o6 }

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 2

Page 198: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-198

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• i1 → o1

– multicast• i1 → {o1, o6 }

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 3

Page 199: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-199

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast

• Crossbar fabric– simple path routing

• element (o,i) turns• i3 → o4

– strictly nonblocking• i1 → o1

– multicast• i1 → {o1, o6 }

i0

i1

i2

i3

i4

i5

i6

i7

o0 o1 o2 o3 o4 o5 o6 o7 4

Page 200: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-200

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast

• Crossbar switch points enable native multicast– switch points duplicate packet flow

• electronic and some optical (but not MEMs mirrors)

problem?

Page 201: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-201

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast Blocking

• Crossbar switch points enable native multicast– switch points duplicate packet flow

• electronic and some optical (but not MEMs mirrors)

– but fabric no longer nonblocking– sets of multicast output groups interfere with one another

Page 202: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-202

© James P.G. SterbenzITTC

Crossbar Switch MulticastNative Multicast Blocking

• Crossbar switch points enable native multicast– switch points duplicate packet flow

• electronic and some optical (but not MEMS mirrors)

– but fabric no longer nonblocking– sets of multicast output groups interfere with one another

• Input shouldn’t wait for free output set– very poor utilisation

Page 203: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-203

© James P.G. SterbenzITTC

Crossbar Switch MulticastOutput Arbitration

• Crossbar switchpoints enablenative multicast

• Output arbitrationneeded to resolveincompatible output sets– fanout splitting:

not all copies needxmit simultaneously

– at cost of implementation complexity

o1 o2 o3 o4 o5

1_3_5

_2345

1234_

_23_5

_2_4_

1 2 1 12

3

3 3 3

24

4

4

5 2

5

i1

i2

i3

i4

i5

Page 204: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-204

© James P.G. SterbenzITTC

Multistage Switch Multicast2×2 Switch Element Duplication

• States– point-to-point

• straight• cross

– multicast• Types

– buffered or unbuffered

– self routing orexternally controlled

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross duplicate

0

Page 205: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-205

© James P.G. SterbenzITTC

Multistage Switch Multicast2×2 Switch Element Duplication

• States– point-to-point

• straight• cross

– multicast• Types

– buffered or unbuffered

– self routing orexternally controlled

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross duplicate

1

Page 206: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-206

© James P.G. SterbenzITTC

Multistage Switch Multicast2×2 Switch Element Duplication

• States– point-to-point

• straight• cross

– multicast• Types

– buffered or unbuffered

– self routing orexternally controlled

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross duplicate

2

Page 207: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-207

© James P.G. SterbenzITTC

Multistage Switch Multicast2×2 Switch Element Duplication

• States– point-to-point

• straight• cross

– multicast• Types

– buffered or unbuffered

– self routing orexternally controlled

control headerdecode

header decode

delay

cut-through delay

cut-through

i0

i1 o1

o0

packet buffer

packet buffer

output multiplexor

straight cross duplicate

3

Page 208: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-208

© James P.G. SterbenzITTC

Multistage Switch MulticastNative Multicast Stages

• Multistage multicast– switch elements

duplicate late• 2×2: replicate

state• n×n internal

– alternative:recirculation

– multiple fabrics• copy stages• cid translation• routing stages

i0

i2

i4

i6

i8

i10

i12

i14

i1

i3

i5

i7

i9

i11

i13

i15

o0

o2

o4

o6

o8

o10

o12

o14

o1

o3

o5

o7

o9

o11

o13

o151110

copy stages routing stages

1010

00000100

translate

Page 209: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-209

© James P.G. SterbenzITTC

Switches and RoutersSW.4 Fast Datagram Switches

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switches

SR.4.1 Overall architecture and performance

SR.5 Programmable, active, & higher layer processing

Page 210: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-210

© James P.G. SterbenzITTC

Fast Datagram SwitchesMotivation

• Connection-oriented fast packet switching– emerged in ATM standards, but ATM failed

• IP became waist of global network infrastructure– increased processing capability enabled fast IP lookups– apply fast packet switching to IP datagram forwarding

Fast Datagram Switch Principle S-1Bd

There is compelling reason to perform high-speed connectionless datagram switching. Apply connection-oriented fast packet switching techniques to fast datagram switching, and exploit technology advances to implement the necessary additional input and output processing in the critical path.

Page 211: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-211

© James P.G. SterbenzITTC

Fast Datagram SwitchesArchitecture

• Fast packetswitch core

• Input processing– IP lookup– packet

classification

• Outputprocessing– packet

scheduling• fair queueing

input processing

management routing and signalling

switch fabric

classify

output processing

output scheduling

link

headerupdate

prefixesinput

processor

link

prefixesinput

processor

link

output scheduling

link

switch fabric

control

Page 212: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-212

© James P.G. SterbenzITTC

Fast Datagram SwitchesArchitecture: Shared Forwarding

input processing • Fast packet switch core• Input processing

– packet classificationmust be performed at input

• Shared forwarding engines– flexible allocation, but– uses switch fabric

ports and bandwidth

• Output processing– packet scheduling

• fair queueing

routing and signalling

switch fabric

output processing

output scheduling

link

prefixes

header processing

prefixes

header processing

input processor

link

headers

input processor

link

headers

output scheduling

link

forwarding engines

switch fabric

control

Page 213: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-213

© James P.G. SterbenzITTC

Datagram Routing and SwitchingExample5.6 IP Packets

• 32 bit IP addresses– host or router interface

• Destination address– used by forwarding– longest-match

lecture IO

04 total lengthIHL TOSfragment id

TTL protocol header checksumsource address

destination addressoptions

(= hl – 20B)

payload(= length – hl – 20B)

flag frag offset

Page 214: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-214

© James P.G. SterbenzITTC

Fast Datagram SwitchesThroughput

• Packet processing rate critical– packet processing must sustain at least average rate– critical path must sustain peak line rate for min size packets

Packet Processing Rate S-II.4p

The packet processing rate (packets per second) is a key throughput measure of a switch. Packet processing software and shared parallel hardware resources must be able to sustain the average packet processing rate. Functions in the serial critical path must be designed for the worst case packet processing rate of the path to avoid queueing and blocking of subsequent packets.

Page 215: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-215

© James P.G. SterbenzITTC

Switches and RoutersSR.5 Programmable & Higher Layer Processing

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

SR.5.1 Alternative technologies and network processorsSR.5.2 Active network nodes

Page 216: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-216

© James P.G. SterbenzITTC

Switches and RoutersSR.5.1 Alternative Technologies

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

SR.5.1 Alternative technologies and network processorsSR.5.2 Active network nodes

Page 217: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-217

© James P.G. SterbenzITTC

Switch ImplementationHardware Alternatives

• Alternative implementation technologies– hardware (fast but inflexible)

• custom VLSI vs. semicustom vs. standard• technology

– DRAM vs. SRAM vs. CAM– CMOS vs. GaAs

• electronic vs. optical

Functional Partitioning and Assignment PrincipleCarefully determine what switch functionality is to be implemented in scarce or expensive technology. S-1C

Page 218: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-218

© James P.G. SterbenzITTC

Switch ImplementationProgrammable Alternatives

• Alternative implementation technologies– fixed hardware (fast but inflexible)– programmable hardware (e.g. FPGAs)– software: flexible

• switch control CPU• general purpose embedded controller • network processor :

specialised embedded controller for packet processing

Page 219: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-219

© James P.G. SterbenzITTC

Switch ImplementationProgrammable Input/Output Processing

• Network processors– compromise among cost, flexibility, performance

• Advantages of input/output programmability– reduce switch design and debug time– allow field upgrades– allow service providers to deploy new protocols/services– enable per port active networking

Page 220: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-220

© James P.G. SterbenzITTC

Switch ImplementationProgrammable Input/Output Processing

• NPs replace– input processing

• lookup• classification

– output processing• scheduling

– additional• encryption• policing

– large buffers• off NP chip

input NPs and memory

management routing and signalling

switch fabric

classify

output NPs and memory

output scheduling

link

headerupdate

prefixesinput

processor

link

prefixesinput

processor

link

output scheduling

link

switch fabric control

Page 221: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-221

© James P.G. SterbenzITTC

Higher Layer and Active ProcessingMotivation and Strategies

• Example motivation for processing in network nodes– firewalls– Web caches– active transcoders– multimedia stream mixing and merging

• Alternative strategies– application layer proxies– active network nodes– layer 4 and layer 7 switches– integration of function into layer 3 forwarding code

Page 222: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-222

© James P.G. SterbenzITTC

Higher Layer and Active ProcessingPerformance

• Active processing reasonable tradeoff– performance vs. flexibility

• Active processing must – not impede the normal fast forwarding path– occur with enough throughput so active queues don’t build

Active Network Processing S-II.4a

Active network processing should not impede the non-active fastpath; packet filters in the critical path must operate at line rate to pass non-active packets. The ability to perform per packet active processing requires sufficient processing power tosustain the require active packet throughput

Page 223: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-223

© James P.G. SterbenzITTC

Active ProcessingGranularity and Node Architecture

• Processing granularity– global control plane

• e.g. routing algorithm

– per flow control plane• e.g. modify RSVP

control

– per packet control plane• e.g. congestion control

– per packet data plane• e.g. transcoding

forwarding

pf

portAAs

portμOS

output processing input processing

routing and signalling

switch fabric

forwarding

pf

portAAs

portμOS

active node processing

NodeOS

MEE

portAAs

portμOS

portAAs

portμOS

switch control

Page 224: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-224

© James P.G. SterbenzITTC

Switches and RoutersSR.5.2 Active Network Nodes

SR.1 OverviewSR.2 Fast packet switchesSR.3 Switch fabric architectureSR.4 Fast datagram switchesSR.5 Programmable, active, & higher layer processing

SR.5.1 Alternative technologies and network processorsSR.5.2 Active network nodes

Page 225: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-225

© James P.G. SterbenzITTC

Active ProcessingNetwork Processors

• Network processors– enable per port

dynamic programmability

NP

forwarding

pf

portAAs

portμOS

output processing input processing

routing and signalling

switch fabric

NP

NP

forwarding

pf

portAAs

portμOS

active node processing

NodeOS

MEE

NP

portAAs

portμOS

portAAs

portμOS

switch control

Page 226: ITTC High-Performance Networkingjpgs/courses/hsnets/lecture-switching-display.pdf · High-Performance Networking The University of Kansas EECS 881 ... • Data manipulation – per

1 October 2008 KU EECS 881 – High-Speed Networking – Switching HSN-SR-226

© James P.G. SterbenzITTC

High-Performance NetworkingAcknowledgements

The material in these foils comes from the textbook supplementary materials:

• Sterbenz & Touch,High-Speed Networking:A Systematic Approach toHigh-Bandwidth Low-Latency Communicationhttp://hsn-book.sterbenz.org