CS

E 2

22A

CS

E 2

22A

Gra

duat

e N

etw

orki

ngG

radu

ate

Net

wor

king

Win

ter

2003

Win

ter

2003

Lect

ure

10: H

igh

Lect

ure

10: H

igh --

Perf

orm

ance

Rou

ter D

esig

nPe

rfor

man

ce R

oute

r Des

ign

Stef

an S

avag

eSt

efan

Sav

age

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Ove

rvie

wO

verv

iew

Rou

ter b

asic

sIn

terc

onne

ctio

n ar

chite

ctur

e◆

Inpu

t Que

uing

◆O

utpu

t Que

uing

◆V

irtua

l out

put Q

ueui

ng◆

Sch

edul

ing

Futu

re b

ottle

neck

sC

ase

Stud

ies

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Wha

t’s

in a

rou

ter?

Wha

t’s

in a

rou

ter?

Phys

ical

com

pone

nts

◆O

ne o

r mor

e in

put i

nter

face

sth

at re

ceiv

e pa

cket

s◆

One

or m

ore

outp

ut in

terf

aces

that

tran

smit

pack

ets

◆A

cha

ssis

(box

+ p

ower

) to

hold

it a

ll

Func

tions

◆Fo

rwar

dpa

cket

s◆

Dro

ppa

cket

s (c

onge

stio

n, s

ecur

ity, Q

oS)

◆D

elay

pack

ets

(QoS

)◆

Tran

sfor

mpa

cket

s? (E

ncap

sula

tion,

Tun

nelin

g)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Wha

t a

rout

er d

oes:

Wha

t a

rout

er d

oes:

the

norm

al c

ase

the

norm

al c

ase

Rec

eive

inco

min

g pa

cket

from

link

inpu

t int

erfa

ceLo

okup

pac

ket d

estin

atio

n in

forw

ardi

ng ta

ble

◆(d

estin

atio

n, o

utpu

t por

t(s))

Valid

ate

chec

ksum

, dec

rem

ent t

tl, u

pdat

e ch

ecks

um

Buf

fer p

acke

t in

inpu

t que

ueSe

nd p

acke

t to

outp

ut in

terf

ace

(inte

rfac

es?

Mca

st)

Buf

fer p

acke

t in

outp

ut q

ueue

Send

pac

ket t

o ou

tput

inte

rfac

e lin

k

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Wha

t a

rout

er lo

oks

like?

Wha

t a

rout

er lo

oks

like?

Cis

co 2

500

Link

sys

DE

FSR

81

19”

8”

Cap

acity

: <10

Mbp

sC

apac

ity: <

10M

bps

1.75

”

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Wha

t a

rout

er lo

oks

like

(2)

Wha

t a

rout

er lo

oks

like

(2)

Cis

co G

SR

124

16Ju

nipe

rM16

0

6ft

6ft

1919””

2ft

2ft

Cap

acit

y:

Cap

acit

y:

160G

b/s

160G

b/s

Pow

er: 4

.2kW

Pow

er: 4

.2kW

3ft

3ft

2.5f

t2.

5ft

1919””

Cap

acit

y:

Cap

acit

y:

80G

b/s

80G

b/s

Pow

er: 2

.6kW

Pow

er: 2

.6kW

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Wha

t a

rout

er lo

oks

like

(3)

Wha

t a

rout

er lo

oks

like

(3)

Plu

risTe

rapl

ex20

w/7

Rac

ks

Cap

acit

y:

Cap

acit

y:

>1T

b/s

>1T

b/s

Pow

er: 4

5kW

P

ower

: 45k

W

(~25

0 ho

mes

?)(~

250

hom

es?)

1 ro

om

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Hig

hH

igh

-- per

form

ance

rou

ters

perf

orm

ance

rou

ters

Gea

red

to c

ore

and

dist

ribut

ion

serv

ice

need

s◆

Req

uire

men

ts: h

igh

spee

d &

hig

h de

nsity

Why

do

we

care

?◆

Moo

re’s

Law

slo

wer

than

link

spe

ed g

row

th (a

nd B

W d

eman

d)»

OC

48c

(2.5

Gbp

s), n

ow, 1

28ns

/pac

ket

»O

C19

2c (1

0Gbp

s), i

n de

ploy

men

t, 33

ns/p

acke

t»

OC

768c

(40G

bps)

, 200

2-3,

8ns

/pac

ket

◆N

eed

high

den

sity

/low

pow

er to

man

age

PO

P c

ompl

exity

»$2

0-10

0k &

2-4

00W

per

por

t, 50

% p

orts

freq

uent

ly fo

r in

tern

al c

onne

ctiv

ity»

DW

DM

can

hel

p w

ith th

e fo

rmer

, but

requ

ires

mor

e in

terfa

ces

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Func

tion

al a

rchi

tect

ure

Func

tion

al a

rchi

tect

ure

Res

erva

tion/

Adm

issi

on

Con

trol

Rou

ting

Prot

ocol

s

Rou

ting

Tabl

eC

lass

ifica

tion

Rul

es

Fire

wal

l

Pack

etC

lass

ifica

tion

Switc

hing

Forw

ardi

ngTa

ble

Out

put

Sche

dulin

g

Con

trol

Pla

ne•C

ompl

ex•P

er-c

ontro

l act

ion

•May

be

slow

Dat

a pl

ane

•Sim

ple

•Per

-pac

ket

•Mus

t be

fast

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Pac

ket

clas

sifi

cati

onP

acke

t cl

assi

fica

tion

Forw

ardi

ng◆

Long

est p

refix

mat

ch o

f des

tinat

ion

agai

nst f

orw

ardi

ng ta

ble

◆R

etur

ns (o

utpu

t por

t, N

ext-h

op M

AC

hea

der)

tupl

e◆

Key

issu

e: fo

rwar

ding

tabl

e gr

owth

◆G

eorg

e w

ill ta

lk a

bout

this

nex

t tim

e

QoS

tagg

ing

◆C

erta

in tr

affic

tagg

ed w

ith h

ighe

r prio

rity

◆P

er fl

ow (s

rcip

, src

port,

dst

ip, d

stpo

rt), p

re s

ourc

e or

des

tpr

efix

, per

pro

toco

l (N

apst

er, e

tc…

)

Fire

wal

l rul

es◆

Blo

ck a

cces

s to

TC

P p

acke

ts w

ith d

stpo

rt !=

80

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Inte

rcon

nect

arc

hite

ctur

eIn

terc

onne

ct a

rchi

tect

ure

Inpu

t & o

utpu

t co

nnec

ted

via

switc

h fa

bric

Kin

ds o

f sw

itch

fabr

ic◆

Bus

◆C

ross

bar

◆S

hare

d M

emor

y

How

to d

eal w

ith

tran

sien

t con

tent

ion?

◆In

put q

ueui

ng◆

Out

put q

ueui

ng◆

Com

bina

tion

Inpu

tO

utpu

tSw

itch

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Firs

t G

ener

atio

n R

oute

rsFi

rst

Gen

erat

ion

Rou

ters

CPU

Rou

teTa

ble

Buf

fer

Mem

ory

MA

C

Line

Car

d

Shar

ed B

us(s

)

MA

C

Line

Car

d

MA

C

Line

Car

d

•Sin

gle

CPU

and

shar

ed m

emor

y;•A

ll cl

assi

ficat

ion

by

mai

n C

PU

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Sec

ond

Gen

erat

ion

Rou

ters

Sec

ond

Gen

erat

ion

Rou

ters

CPU

Rou

teTa

ble

MA

C

Line

Car

d

Shar

ed B

us(s

)

Forw

ardi

ngC

ache

Buf

fers

MA

C

Line

Car

d

Forw

ardi

ngC

ache

Buf

fers

MA

C

Line

Car

d

Forw

ardi

ngC

ache

Buf

fers

Dire

ct D

MA

on

cach

e hi

t

Cac

he o

fre

cent

rout

es

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Thi

rd G

ener

atio

n R

oute

rsT

hird

Gen

erat

ion

Rou

ters

MA

C

Line

Car

d

Forw

ardi

ngTa

ble

Buf

fers

CPU

Car

d

Rou

ting

Tabl

e

CPU

MA

C

Line

Car

d

Forw

ardi

ngTa

ble

Buf

fers

Switc

h Fa

bric

•Sha

red

inte

rcon

nect

(usu

ally

cro

ssba

r)•C

entr

aliz

ed s

ched

uler

•Ful

l for

war

ding

tabl

ein

line

car

d•F

ixed

cel

ls (w

hy?)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Out

put

queu

ing

Out

put

queu

ing

Out

put i

nter

face

s bu

ffer

pack

ets

Pro

◆S

impl

e al

gorit

hms

◆S

ingl

e co

nges

tion

poin

t

Con

◆N

inpu

ts m

ay s

end

to th

e sa

me

outp

ut◆

Req

uire

s sp

eedu

p of

N

Inpu

tO

utpu

tSw

itch

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Inpu

t qu

euin

gIn

put

queu

ing

Inpu

t int

erfa

ces

buffe

r pa

cket

s

Pro

◆S

ingl

e co

nges

tion

poin

t◆

Sim

ple

to d

esig

n al

gorit

hms

Con

◆M

ust i

mpl

emen

t flo

w c

ontro

l◆

Low

util

izat

ion

due

to H

ead-

of-L

ine

(HoL

) Blo

ckin

g »

Util

ilim

ited

to 2

-2^

.5=5

8%

Inpu

tO

utpu

tSw

itch

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Hea

dH

ead

-- of

of-- L

ine

Blo

ckin

gLi

ne B

lock

ing

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Vir

tual

Out

put

Que

ues

Vir

tual

Out

put

Que

ues

(cou

rtes

y N

ick

McK

eow

n)(c

ourt

esy

Nic

k M

cKeo

wn)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

IQ +

Vir

tual

Out

put

Que

uing

IQ +

Vir

tual

Out

put

Que

uing

Inpu

t int

erfa

ces

buffe

r pa

cket

s in

per

-out

put

virt

ual q

ueue

s

Pro

◆S

olve

s bl

ocki

ng p

robl

emC

on◆

Mor

e re

sour

ces

per p

ort

◆C

ompl

ex a

rbite

r◆

Stil

l lim

ited

by in

put/o

utpu

t co

nten

tion

(sch

edul

er).

◆R

R: 1

=1/e

= 6

3%

Inpu

tO

utpu

tSw

itch

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Sw

itch

sch

edul

ing

Sw

itch

sch

edul

ing

Prob

lem

◆M

atch

inpu

ts a

nd o

utpu

ts◆

Res

olve

con

tent

ions

◆N

o ou

tput

pac

ket d

rops

◆M

axim

ize

thro

ughp

ut◆

Do

it in

con

stan

t tim

e…M

any

algo

rithm

s fo

r uni

form

traf

fic a

ssum

ptio

n◆

E.g

. TD

M, M

axim

um s

ize

bipa

rtite

mat

ch◆

App

roxi

mat

e an

swer

s (e

.g. i

SLI

P, s

ubm

axim

alm

atch

)R

ecen

t res

ult (

Dai

et a

l, 20

00)

◆M

axim

al s

ize

mat

chin

g +

spee

dup

of tw

o gu

aran

tees

10

0% u

tiliz

atio

n fo

r mos

t tra

ffic

assu

mpt

ions

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Mod

ern

rout

erM

oder

n ro

uter

IQ +

VoQ

+ O

Q◆

Spe

edup

of 2

◆C

entra

l sch

edul

er◆

Fixe

d-si

zed

inte

rnal

cel

ls

Pro

◆C

an a

chie

ve u

tiliz

atio

n of

1◆

Can

sca

le to

mul

tiple

Tb/

s

Con

◆M

ultip

le c

onge

stio

n po

ints

◆C

ompl

exity

Inpu

tO

utpu

tSw

itch

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Typ

ical

fun

ctio

n br

eakd

own

Typ

ical

fun

ctio

n br

eakd

own

Inpu

t int

erfa

ce◆

Forw

ardi

ng◆

Virt

ual o

utpu

t que

uing

Switc

h◆

Sch

edul

ing

inpu

t int

erfa

ce re

ques

ts to

out

put i

nter

face

s◆

Mul

ticas

t sch

edul

ing

Out

put i

nter

face

◆Q

ueue

pac

kets

for t

rans

mis

sion

◆C

lass

ifica

tion

◆B

uffe

r man

agem

ent (

whi

ch p

ktto

dro

p)◆

Sch

edul

ing

(whi

ch p

ktto

sen

d fro

m b

uffe

r)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Nex

t bo

ttle

neck

sN

ext

bott

lene

cks

Buf

ferin

g at

hig

h sp

eed

◆S

RA

M d

ensi

ty to

o lo

w fo

r BW

*D o

f 40G

bps

link

◆D

RA

M to

o sl

ow◆

SR

AM

mem

ory

man

agem

ent a

s ca

che

for D

RA

M

Sche

dule

r and

arb

iter o

verh

ead

◆Li

mits

siz

e of

sw

itch

and

link

BW

◆Tw

o-st

ate

switc

h (C

hang

et a

l, 20

00);

no s

ched

uler

Hig

h de

nsity

(100

’s-1

000’

s of

line

car

ds)

◆P

hysi

cal d

ista

nce

to s

uppo

rt de

nsity

; ele

ctric

al li

nks

degr

ade

◆O

ptic

al li

nks;

opt

ical

cro

ss c

onne

ct (M

EM

s,tu

nabl

ela

sers

)

Tim

e to

mar

ket,

Pow

er/H

eat

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Cas

e st

udy

Par

trid

ge e

t al

98

Cas

e st

udy

Par

trid

ge e

t al

98

IQ +

VO

Q +

OQ

15 li

ne-c

ards

eac

h w

ith u

p to

16

inte

rfac

es◆

Forw

ardi

ng e

ngin

es s

epar

ate

from

lc(w

hy?)

Poin

t-to-

Poin

t sw

itch

◆50

Gbp

s ca

paci

ty, 2

5% lo

st to

ove

rhea

d◆

32M

pps

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Rou

ter

Arc

hite

ctur

eR

oute

r A

rchi

tect

ure

pack

ethe

ader

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Impl

emen

tati

onIm

plem

enta

tion

Line

car

ds◆

53M

hz F

PG

A, I

/O p

roce

ssin

g, q

ueue

man

agem

ent

Forw

ardi

ng e

ngin

e◆

430M

hz 2

1164

Alp

ha u

sing

L2

as ro

ute

cach

e, L

3 ho

lds

all

»12

,000

rout

es..

clai

m 9

5% h

it ra

te…

reas

onab

le?

◆Lo

ts o

f pai

n w

ith u

PI/O

bei

ng n

ot q

uite

righ

t◆

Doe

sn’t

chec

k IP

che

cksu

m.

Is th

is o

k?Sw

itch

◆S

ched

uler

FP

GA

Con

trol

pla

ne◆

Net

BS

Dru

nnin

g G

ateD

on 2

33-M

hz 2

1064

◆P

ushe

s ro

utin

g ta

bles

to F

Es

and

hand

les

exce

ptio

nal

pack

ets

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Sum

mar

y: I

nnov

atio

nsS

umm

ary:

Inn

ovat

ions

1.Ea

ch F

E ha

s a

com

plet

e se

t of t

he ro

utin

g ta

bles

2.A

sw

itche

d fa

bric

is u

sed

inst

ead

of th

e tr

aditi

onal

sha

red

bus

3.FE

s ar

e on

boa

rds

dist

inct

from

the

line

card

s4.

Use

of a

n ab

stra

ct li

nk la

yer h

eade

r5.

Incl

ude

QoS

pro

cess

ing

in th

e ro

uter

Que

stio

ns◆

Wou

ld y

ou u

se a

mic

ropr

oces

sor f

or fo

rwar

ding

toda

y?◆

How

impo

rtant

is p

rogr

amm

abili

ty?

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Cas

e st

udy:

McK

eow

n et

al9

8C

ase

stud

y: M

cKeo

wn

et a

l98

Tiny

Ter

ra: a

n in

spira

tion

for C

isco

GSR

(nee

BFR

)IQ

+ V

OQ

+ O

Q

32 in

terf

aces

at 1

0Gbp

s ea

ch32

x32

cros

sbar

(320

Gbp

s ag

greg

ate)

64bi

t cel

lsLi

ne c

ards

ASI

C-b

ased

iSLI

Psc

hedu

ling

algo

rithm

◆G

reed

y, s

ubm

axim

albi

parti

te m

atch

ing

◆E

asy

to im

plem

ent

(2N

arb

iters

)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Con

clus

ion

Con

clus

ion

It is

feas

ible

to b

uild

hig

h sp

eed

rout

ers

◆40

Gbp

s lin

k sp

eeds

◆10

-20

Tbps

aggr

egat

e ca

paci

ty

But

…◆

Lim

ited

prog

ram

mab

ility

◆H

igh

com

plex

ity, s

low

tim

e to

mar

ket

»Ju

nipe

r I2

AS

IC 2

.5M

gat

es»

Typi

cal O

C19

2 LC

~30

M g

ates

! »

Sta

rting

to re

quire

sig

nific

ant o

n-ch

ip S

RA

M◆

Nex

t gen

(OC

3072

160

Gbp

s LC

) may

be

clos

e to

cro

ss-o

ver

poin

t for

CM

OS

(luc

kily

, not

cle

ar th

ere

is d

eman

d an

yway

)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Ope

n is

sues

Ope

n is

sues

Net

wor

k pr

oces

sors

◆

Qui

ck T

TM +

hig

h pe

rform

ance

◆A

pplic

atio

n-sp

ecifi

c pr

ogra

mm

abili

ty (f

or h

igh-

touc

h fe

atur

es)

Softw

are

desi

gn◆

Neg

lect

ed, b

ut c

ritic

al

Opt

ical

sw

itchi

ng◆

Look

s lik

e ci

rcui

t sw

itchi

ng, v

ery

low

pow

er◆

Ok

to lo

se p

acke

t sw

itch

effic

ienc

y if

BW

is c

heap

and

relia

ble

◆C

urre

ntly

ver

y ex

pens

ive

to li

ght a

lam

bda

(am

ps, l

aser

s)

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

For

next

tim

e…Fo

r ne

xt t

ime…

Geo

rge

Varg

hese

will

be

pres

entin

g on

IP L

ooku

ps

Ther

e is

a p

acke

t loo

kup

surv

ey p

aper

to re

ad th

at

will

be

post

ed to

nigh

t.

Febr

uary

18,

200

3C

SE 2

22A

--Le

ctur

e 10

–R

oute

r Des

ign

Som

e sl

ides

cou

rtes

y of

Nic

k M

cKeo

wn

Poi

nts

of P

rese

nce

(P

oint

s of

Pre

senc

e ( P

OP

sP

OP

s ))

A B C

POP1

POP3

POP2

POP4

D E F

POP5

POP6

POP7

POP8