exterior routing 201 howard c. berkowitz hberkowi@nortelnetworks.com hcb@clark.net (703)998-5819 esn...

Exterior Routing 201Exterior Routing 201

Howard C. BerkowitzHoward C. Berkowitzhberkowi@nortelnetworks.com hberkowi@nortelnetworks.com

hcb@clark.nethcb@clark.net

(703)998-5819 ESN 451-5819(703)998-5819 ESN 451-5819

NANOG 21 Exterior Routing tutorial 2/17/20012

AgendaAgenda

• What's the problem?— Formal and informal clue

— ISP service offerings

• Quirks, Defnitions, and Issues

• ISP External Scenarios

• POP and other infrastructure

• Router requirements

• Playing in the Club

• Turning it On

If there's time...full employment for consultants: path selection

What is the Problem to be What is the Problem to be Solved?Solved?

Good little boys and girlsread RFC1771

and live happily ever after

• Noah.

• (yawn) MMMmmmmhp?

• Noah.

• Yeahh?

• Build an ISP.

ISPs Facing End UserISPs Facing End User

• Entry— Basic Internet Access

— Hosting

— Availability and QoS

— Dealing with specialized access providers (DSL, CATV, etc.)

— Dealing with content providers

— Voice services?

• Improvement for Users– Improving capacity– Improving availability– Adding services– Perceptions of end-to-end

Before the AnimalsBefore the Animals

Uplinks

Routers

User Hosts

Downlinks

Management

Facilities

HVAC Staff

Load the ArkLoad the Ark

Policies

Traffic

From Downstreams

Policies

Traffic

From Upstreams

From Users

Traffic AAA

Traffic

From Virtual Hosts

Quirks, Definitions and Quirks, Definitions and IssuesIssues

I said "peer," not "peer"I said "peer," not "peer"

• Peer relationship 1— Basic BGP session

• Peer relationship 2— Mulual benefit customers reach one another

— No monetary exchange

— Each advertises customer routes

• Transit Provider relationship— Customer pays for service

— Full routes available to customer

(C) O'Leary Museum and Library Association Ltd. Inc.

Closest Exit RoutingClosest Exit RoutingHot potatoHot potato

• Paths are not optimized end-to-end

• Paths are optimized for each AS

Asymmetrical RoutingAsymmetrical Routing

• No guarantee that traffic leaving your AS at one point

• Will return at the same point

• Remember

—Each AS in both directions makes decisions on its information

ISP ISP ScenariosScenarios

Basic Internet Access ISPBasic Internet Access ISP

POP2 POP3

POP1HostedServers

InternalServers

ISP #2 ISP #1

/18 /18 /18

/16/16

To 70-90% of customers Default routeTo 5-10% of customers Partial routesTo 10% of customers Full routes

From customersFew # public routes ??? VPN

Bilateral PeeringBilateral Peering

BigISP 1

BigISP 2

eBGP Relationship

Exchange of customer routes only Some aggregation No infrastructure routes

Highest bandwidth requirement

"Tier 1 Provider" Does not buy transit service from anyone Has default-free routers Gets all routes from bilateral/multilateral peering Total RIB size of 1.3-1.5 * DefaultFreeZone (D)

Large Content ProviderLarge Content Provider

10/100

Server

L4 distributionGE ports

Firewall,etc.

Provider

Server Server

L7 Distribution

L3 Path Determination• Sometimes bandwidth limited

• Provider may be default free

• Often high touch processing limited

• Possible SLA and VPN agreements

May participate in content distribution, caching

Layer2

Fabric

ISP 6RouteServer

Multilateral PeeringMultilateral PeeringeBGP Relationships

Depending on exchange rules Exchange of customer routes only Most common case Some aggregation No infrastructure routes Some ISPs buy transit services Can receive full routes Private peerings

Largest carriers tend to avoid due tocongestion

ISPs can peer with route server rather than a mesh of ISPs May be done to reduce BGP peers Or simply for statistics collection

Special Case: Local ExchangesSpecial Case: Local Exchanges

• Entry— Who's in charge?

— Connectivity

— Facilities

— Allow content providers?

— Allow end users?

— Peering model?

— Supplementary services?

• Improvements

Layer2? 3?Fabric

ISP 6RouteServer

POP and Other Internal POP and Other Internal DesignDesign

Typical Basic POP ImplementationTypical Basic POP Implementation

Gigabit Ethernet Frame Interfaces ATM Interfaces

LAN Switch

ManagementServers

AccessServer

DialupCustomers

Router Fabric

ISP CoreRouter 1

ISP CoreRouter 2

DedicatedCustomers

CustomerSite Routers

FrameDS3

32x/30

DedicatedCustomers

CustomerSite Router

FullDS3

1 per POP25 per POP

450 users per POP

/18 /18

Transit Provider POP, Transit Provider POP, Intra-POPIntra-POP

Design Alternatives 1. POP is a route reflector cluster Core is higher-level cluster

2. Each POP is a private or public AS Full mesh iBGP or route reflectors inside POP Confederation between POPs

3. IGP within POP Controlled redistribution inside POP to BGP Prefer intra-POP of same metric

AccessRouter

POPRouter

Public AS

POP ConfederationsPOP Confederations

POP AS65000 POP AS65111 POP AS65222

Public AS

POP ReflectorsPOP Reflectors

POP AS65000 POP AS65111 POP AS65222

Open Access/Specialized AccessOpen Access/Specialized Access

Layer1/2

FabricSubscribers

ISP 3InternalRoutedNetwork

TunnelServer

ContentServers

InternalRouting

&Switching

Tunneled AddressingTunneled Addressing

Data Provider1

Voice Provider1

EnterpriseVPN NAS

AccessGateway

Data Provider2

L2TP, Differv High

Access OAM address space

PPPoE or GRE

ISP address space

Data 1Data 2VPN

DHCPDNS

Router Router RequirementsRequirements

Big part of the solution...but not Big part of the solution...but not all.all.

Routing ParadigmsRouting Paradigms

Number of Routes

Forwarding Bandwidth

Hello Processing

Number of Interfaces

Policy Analysis

QoS Awareness

Low High Medium

Low Medium High

Medium High Medium

End to End EtE & PHB PHB

Low High Low

L4/7 Processing Medium High Low

Enterprise Edge Core

Observations on Routing Table SizeObservations on Routing Table Size

• Global default-free table continues to grow exponentially— 96509 routes as of Tony Bates' CIDR report 2/11/2001

— Let the default routing table size be D

• Large provider often has 1.3 to 1.5 D active routes— additional routes are more-specific customer & internal

— may also have substantial numbers of inactive routes

Growth in Global Routing Table SizeGrowth in Global Routing Table Size

Sep 01

Sep 02

Sep 03

Sep 04

85K public

Growth in Typical Tier 1 Routing Growth in Typical Tier 1 Routing Table SizeTable Size(external + customer, not infrastructure)(external + customer, not infrastructure)

Sep 01

Sep 02

Sep 03

Sep 04

85K public42K internal

Observation: More than RoutesObservation: More than Routes

• Customer routes

• Paths per route

• Route validity

ConvergenceConvergence

• Global routing system

• Intra-AS

• Single Router

Single Router ConvergenceSingle Router Convergence

• Initialization— Time to add new route

— Time to add better route

— Time to withdraw route

— Time to withdraw and replace route

• Parameters— Matrix: number of peers

versus– Routes advertised– Routes accepted

• Performance Modifiers— Route filtering

— Route flapping

— Packet vs. route filtering

draft-berkowitz-bgpcon-0x.txt

Distinguish among casesDistinguish among cases

• Failover of link or router between customer and provider

• Rerouting to intranet/adjacent provider resources

• Rerouting to arbitrary internet destnation

More multihoming in next tutorial

S-T-R-E-T-C-HS-T-R-E-T-C-H

Joining the Joining the ClubClub

More than Just Addresses, Protocol...More than Just Addresses, Protocol...

Routing Registry

Routeobjects

ASobjects

Maintainerobjects

RoutingSystem

Configs

Customer

SpecifyPolicy

Route Track Service

ISP with Prefixes

Allocate

Directories

ReverseDNS

AddressRegistry

RouteRegistry

ComplexityComplexity

• BGP itself is fairly simple

• Additional attributes it carries are more complex

• Policy actions taken inside router (BGP sender or receiver) far more complex than the protocol itself

"BGP Transmits Policies""BGP Transmits Policies"

OOppeerraattiioonnaall RReellaattiioonnsshhiippss 11

AAddddrreesssseess aanndd DDeelleeggaattiioonn

Addressauthority

Addressdelegation

Prefixes

ReverseDNS

Obtain routable address spaceObtain routable address space

• Apply to registry— RIPE, APNIC, ARIN

— If immediate need for /19 or /20*

• Obtain addresses from upstream ISP— If /19 or /20 cannot be justified

• Registry needs— Network design

— Justification for address space

Origination vs. AdvertisingOrigination vs. Advertising

128.0.0.0/20

/23POP Dialups

/23Internal

/23Customers

/25/25/25/2532*

/28/24 /24/25/25

AS 65000

192.0.0.0/16AS64444

an AS65000Customer

AS65000

128.0.0.0/19

AS65000

192.0.0.0/16

AS64444

Aggregating your Own TrafficAggregating your Own Traffic

128.0.0.0/20

/23POP Dialups

/23Internal

/23Customers

/25/25/25/2532*

/28/24 /24/25/25

AS65000

128.0.0.0/19

Suppress more specific routesunless required by multihoming

Advertising with NO-EXPORTAdvertising with NO-EXPORT

AS6333364.0.0.0/12

Assigns64.0.0.0/22

Assigns64.0.4.0/22

AS62222 AS61111

AS6100096.1.0.0/16

Advertises64.0.0.0/22 NO-EXPORT

Advertises64.0.4.0/22 NO-EXPORT96.1.0.0/16

Aggregation is better than Aggregation is better than AggravationAggravation

• Blackhole routes for your blocks— Avoid more-specifics— Use NO-EXPORT when controlling load to upstream

• Encourage customers to aggregate— Proxy aggregation hard to administer

• Understand which blocks you can advertise— And do ingress/egress filtering

Preparing for Address Request (1)Preparing for Address Request (1)

• Address requirements of services are you offering

• Dynamic addressing— Dialup

— Residential broadband

• Private addressing— Enterprises homed only to you

— Dialup/broadband not offering servers

• Globally addressable

Prepare for Address Request (2) Prepare for Address Request (2) An ISP TopologyAn ISP Topology

POP11 internal LAN

100Dial

8smallLANs

1med.LAN

POP21 internal LAN

100Dial

8smallLANs

1med.LAN

POP31 internal LAN

100Dial

8smallLANs

1med.LAN

POP41 internal LAN

100Dial

8smallLANs

1med.LAN

CoreRouter 1

CoreRouter 2

Hosting Farm 1 Hosting Farm 2InfrastructureServers

Switch

EEssttaabblliisshhiinngg aann AASS ((11))

AASS NNuummbbeerr RReeqquueesstt

• In request to AS number registry— Administrative and technical contacts

— Autonomous system name

— Router description

— Deployment schedule

— Networks (by name) connected by the router(s)

— Internet addresses of the routers

Establishing an AS (2)Establishing an AS (2)Registering in Routing RegistryRegistering in Routing Registry

• Minimum requirements— Maintainer object

— AS object

— Route object (s)

Establishing an AS (3)Establishing an AS (3)Operational deploymentOperational deployment

• Build configuration— Policy implementation

— Ingress/egress filtering

• Establish security procedures

• Start BGP connections

RRoouuttiinngg RReeggiissttrryy OObbjjeeccttss

• Basic— AS

— Route

— Maintainer

• Additional— Inter-AS Network

— Community

— Router

Refinements

OOppeerraattiioonnaall RReellaattiioonnsshhiippss 33::

RReeggiissttrriieess,, DDoommaaiinnss,, eettcc..

Addressauthority

Addressdelegation

Prefixes

Routeobjects

ReverseDNS

Autonomous SystemAutonomous System

• Basis of exterior routing

• AS originate routes for some prefixes they want to be visible

• AS advertise routes to one another— Advertisement may not contain all addresses

— Not all advertisements need be accepted

Current AS DefinitionCurrent AS DefinitionRFC 1930RFC 1930

• Connected group of IP CIDR blocks

• Run by one or more network operators

• Single routing policy — announced to the general Internet

— announced with BGP-4

AASS NNuummbbeerr

• 16 bit number— 32 bit under discussion

• Numbers assigned by registries— Routing policy should be stored in registry

— ISPs can mirror routing registry -- place for sensitive data

• Private ASNs— 64512 through 65535

— Private AS stripping, confederations

OOppeerraattiioonnaall RReellaattiioonnsshhiippss 22::

AAddddeesssseess aanndd AAuuttoonnoommoouuss SSyysstteemmss

Addressauthority

Addressdelegation

Prefixes

ReverseDNS

Full Employment for Consultants: Full Employment for Consultants: Policies are Policies are insideinside Routers Routers

• Advertising Policies— Outbound to other AS

— BGP advertisement sources

— Outbound route filters

— Route must be in internal routing table

• Acceptance Policies— Inbound AS filters

— Inbound route filters

Stop! Stop! What are you going to What are you going to

Advertise?Advertise?•Routes Assigned/Allocated to You

•Routes Assigned/Allocated to Customers

•Routes for which you provide Transit

Advertising AffectsAdvertising Affects

• The way the world sees you/sends to you

• Binary— Routes to which you provide routing

• Quantitative Preferences— Multi-Exit Discriminators to your Neighbors

— AS Path Manipulation to all

Routes Eligible to AdvertiseRoutes Eligible to Advertise

• Are reachable by your IGPor static routes

• Unless they are black holes— Which conceptiually are reachable

• Do not advertise— Spoofed source addresses

— Your internal addresses

— RFC1918 space

— Known rogues?– RBL?

Stop! Stop! What are you going to What are you going to

Accept?Accept?

•It depends

•Only those routes you will do something about

•Otherwise default

Do Not AcceptDo Not Accept

• RFC1918 source or destination

• Unexpected sources not assigned/allocated to peers

• Your internal addresses from peers

Turning it Turning it OnOn

BBGGPP CCoonnffiigguurraattiioonn OOvveerrvviieeww

• Plans and policies first!

• Define system of BGP speakers

• Specific BGP speaker configuration— Identifier

— BGP process

— Neighbors

— NLRI to advertise

— Filters and other policy mechanisms

Cisco commands used as examples

Policy Implementation FlowPolicy Implementation Flow

MainBGPRIB

AdjRIBOut

Outgo-ing

AdjRIBIn

Incom-ing

MainRIB/FIB

Static&

HWInfo

AS21R1

AS21R2

All equivalent from a policy standpoint!

Policy vs. Protocol FlowPolicy vs. Protocol Flow

BGP ConfigurationsBGP Configurations

• Know global information (AS, policies, etc.)

• Establish router ID

• Create BGP process

• Identify internal and external peers

RRoouutteerr IIDD aanndd llooooppbbaacckk iinntteerrffaaccee

interface loopback 0ip address 192.168.0.1 255.255.255.0

RReeffiinniinngg tthhee CCoonnffiigguurraattiioonn

Single and Multiple Links Single and Multiple Links

to a Single Providerto a Single Provider

TThhee BBGGPP TTuunnnneell

Serial 0 Serial 0

Serial 1 Serial 1

Loop 0 Loop 0

ebgp-multihop needed whenneighbor is not on same subnet

LLooaadd BBaallaanncciinngg 11::

IIPP LLeevveell ttoo SSiinnggllee PPrroovviiddeerr RRoouutteerr

Serial 0 Serial 0

Serial 1 Serial 1

Loop 0 Loop 0

CustomerAS

ProviderAS

LLooaadd BBaallaanncciinngg 11::

MMuullttiippllee RRoouutteerrss

CustomerAS

ProviderAS

AAnnootthheerr NNoonn--BBGGPP AAlltteerrnnaattiivvee

OOSSPPFF RRoouuttiinngg DDoommaaiinn

Default Route (0.0.0.0/0) Metric Type 1 Equal Metrics

Static routesD1-A0ASBR1

D1-A0ASBR2

Multiple OSPF DefaultsMultiple OSPF Defaults

ISP 1POP

ISP 2POP

Default Route (0.0.0.0/0) Metric Type 2

Higher Metric to ISP 2 (Backup)

Static routesD1-A0ASBR1

D1-A0ASBR2

Blackhole RouteBlackhole Route

• Establish static route to your block(s) ip route 1.2.3.4 255.255.240.0 null0

• Redistribute/import into BGP

• Suppress more-specific prefix advertising

Effects of BlackholingEffects of Blackholing

• No route flapping outside your AS

—If your internal routes go up or down

• Incoming traffic for specific routes that are down

—Doesn’t match any internal route

—Automatically discarded without concerning anyone else

BGP Path BGP Path SelectionSelection

Next Hop AccessNext Hop Access

Advertised routevia R1

SSccooppee::

MMEEDD vvss.. LLooccaall PPrreeffeerreennccee vvss.. WWeeiigghhtt

Weight

Local Preference

Weight

AS1 AS2

AAddmmiinniissttrraattiivvee WWeeiigghhtt ((CCiissccoo eexxtteennssiioonn))

Rules in this router set R1 weight to 100, R2 weight to 500

Weight exampleWeight examplefor load sharingfor load sharing

PrimaryISP

Default local preference 500All routes ^ AS_Backup + local preference 100

BackupISP

Default local preference 200

TTiieebbrreeaakkeerr ffoorr EEqquuaall WWeeiigghhtt::

LLooccaall PPrreeffeerreennccee

Advertised routevia R1, local preference 100

Advertised routevia R2, local preference 500

Local Preference Local Preference example for load sharingexample for load sharing

PrimaryISP

Default local preference 500All routes ^ AS_Backup + local preference 100

BackupISP

Default local preference 200

PPrreeffeerr llooccaallllyy oorriiggiinnaatteedd rroouutteess

Locally definedvia R2

AS PathAS Path

1 2 4 5

SShhoorrtteesstt AASS PPaatthh ((CCiissccoo eexxtteennssiioonn))

AS AS AS AS Route

AS AS Route

Full Employment For Consultants:Full Employment For Consultants:Interpreting AS PathInterpreting AS Path

• Default assumption: local preference set based on AS_PATH

• Cisco considers it as part of the algorithm

AS Path PrependingAS Path Prepending

• Applies to routes you advertise

• Makes them less attractive to others

• Increases AS_PATH length— your AS put in the path twice

Limitations of PrependingLimitations of Prepending

1 2 4 5

Route Learned from eBGP

Route Learned from iBGP

External Paths PreferredExternal Paths Preferred

RemoteAS

MED=100

MED=500

Lowest MEDLowest MED

Full Employment For Consultants:Full Employment For Consultants:Weight, Local Preference & MEDWeight, Local Preference & MED

• HIGHER value wins— Weight

— Local preference

• LOWER value wins— MED

— Cisco default: route with no MED preferred

— IETF: route with no MED least preferred

Full Employment For Consultants:Full Employment For Consultants:Scope of MEDScope of MED

• Default assumption: — MEDs only compared between exits to the same adjacent AS

• Alternate: always-compare-MED— Useful at exchange points, possibly private peerings

— Cisco knob

CClloosseesstt NNeeiigghhbboorr

IGP metric to R1=100

IGP metric to R1=500

LLoowweesstt BBGGPP rroouutteerr IIDD

R22.2.2.2

R11.1.1.1

exterior routing 201 howard c. berkowitz hberkowi@nortelnetworks.com hcb@clark.net (703)998-5819 esn...

exterior routing tutorial

customer slide

information slide

vpn slide

asymmetrical routing

path selection slide

defaultfreezone d slide

src dest slide

Documents

5819 perencanaan mesin_listrik

gaceta nro 5819 - resolución 327

hcb - who we are

Весы highland тип hcb - vesmarket.ru602h hcb 602 hcb...

hcb-f9nn manual eng

lineamientos técnicos hcb 2015

huileries du congo belge (hcb) service médical en 1928. (d...

january 19-20, 2005ietf 61.5 - redwood city, ca1 lemonade...

intelligent traffic management dale o’grady...

hcb 2013 fundraiser catalog

hcb bridge inspection · pdf filethe hillman-composite beam...

ГОСТ 5819-78 Реактивы. Анилин....

xcon framework overview & issues editors: mary barnes...

june 10, 2004ietf 59,5 - richardson, tx, usa1 lemonade...

disman – ietf 56 alarm mib sharon chisholm...

customer satisfaction 201 howard c. berkowitz...

hcb tools patents catalogue

manual de utilizare hsb / hsb p / hsb e / hbb / hcb / hcb p

hcb-108 en es zs zt 1215 1478 2 -...

hcb thesis 202