Peering Exchange Architectures

Jeff Bartig

University of Wisconsin

WiscNet Engineering

Overview OmniPOP - new CIC R&E exchange in

Chicago Way too many hours of conference calls

about hardware and architecture of OmniPOP

I returned Paul Schopis’ phone call.

Layer 2 Shared Exchange Layer 2 switch Each peer connects to

switch Single broadcast

domain Generally a single

subnet allocated - each peer gets an address out of the block

Used at many commercial exchanges

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.QuickTime™ and a

None decompressorare needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

Layer 2 shared - Pros/ConsCons

Single broadcast domain

Peer abuse possible Broadcast traffic Multicast difficulties Per-peer statistics more

difficult to collect (per-MAC address stats necessary)

Single MTU size

Pros Easy/Simple No exchange provider

involvement needed to establish a new peer

Many peers on a single port - lower cost

Layer 2 VLAN Exchange Each peer has a

trunk interface on layer 2 exchange switch

Each peer may have a VLAN to every other peer

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.QuickTime™ and a

None decompressorare needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

Layer 2 VLAN Exchange Each peer has a

trunk interface on layer 2 exchange switch

Each peer may have a VLAN to every other peer

(n * (n-1)/2 VLANs for a full mesh

QuickTime™ and aNone decompres s or

are needed to see this pic ture.

QuickTime™ and aNone decompres s or

are needed to see this pic ture.

QuickTime™ and aNone decompres s or

are needed to see this pic ture.

QuickTime™ and aNone decompres s or

are needed to see this pic ture.QuickTime™ and a

None decompres s orare needed to see this pic ture.

QuickTime™ and aNone decompres s or

are needed to see this pic ture.

Layer 2 VLAN - Pros/ConsPros

Separation MTU per VLAN Multicast easier Many peers on a single

port - lower cost Logical interface stats

Cons Many VLANs VLAN conflicts

possible Exchange provider

involvement if VLANs are not preallocated

Layer 1 No central exchange

hardware Each peer

establishes cross connects to parties they want to peer with

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.QuickTime™ and a

None decompressorare needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

Layer 1 Pros/Cons Requires an

interface for each peer

No exchange provider hardware needed - may be a cost savings

Capacity/protocol flexibility

Layer 3 Exchange Each peer gets a

port on the exchange router

Each peer establishes a BGP session with exchange router

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.QuickTime™ and a

None decompressorare needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

Layer 3 - Pros/ConsCons

Less control Routing policy

determined by exchange provider

Extra AS hop in path, possible impact on routing decisions

Pros Simpler Single BGP session No need for a

peering coordinator - outsourcing to exchange provider

Layer 3 Exchange - AS-path Length Concern Example

Peer to WN AS PathsLayer 2 Exchange

2381 701 2381 WN direct peer shorter

Layer 3 Exchange 54321 2381 701 2381 Equal paths. Which one will be

chosen by peer?

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

AS-Path Length Experiment WiscNet peers with 31 networks at Equinix-

Chicago What difference would it make if we instead

did this peering via a layer 3 exchange? Depends upon the routing policies of the peers (local pref wins over AS-path length)

Prepended extra AS hops into the advertisements to see what would happen.

AS-Path Length Experiment Results

13:50 - prepended extra hop

450 to 350Mb/s drop 22% loss

14:26 - prepended 2nd extra hop

410 to 350Mb/s drop additional 15% loss

14:45 - removed all prepending

310 to 430Mb/s increase 38% increase

End