inter-domain routing lecture

75
15744 - Fall 2004 Lecture 3 1 Inter-Domain Routing: BGP, Overlay Routing, Multihoming Vyas Sekar Based on slides from: Srini Seshan, Tim Griffin

Upload: graham-costelloe

Post on 04-Jun-2018

223 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 1/74

15744 - Fall 2004 Lecture 3 1

Inter-Domain Routing:

BGP, Overlay Routing,

Multihoming

Vyas Sekar

Based on slides from:Srini Seshan, Tim Griffin

Page 2: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 2/74

15744 - Fall 2004 Lecture 3 2

Readings

•  Assigned – [Mit notes] – Overview of BGP/ Interdomain routing

 – [Gao01] – Inferrring AS relationships

 – [Lab00] – BGP Convergence

 – [S+99] -- Suboptimality in Internet routing• Optional

 – [Griffin01] – BGP Tutorial

 – [SARK01] – Characterizing the Internet Hierarchy from multiplevantage points

 – [APMS+04] – A Comparison of Overlay routing and multihoming – [AWBKM] – Resilient Overlay Networks

 – [GW02] – iBGP configuration

Page 3: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 3/74

15744 - Fall 2004 Lecture 3 3

Outline

• Need for hierarchical routing

• BGP –  ASes, Policies

 – BGP Attributes

 – BGP Path Selection – iBGP

 – Inferring AS relationships

• Problems with BGP – Convergence

 – Sub optimal routing

• Overlay Routing and Multihoming

• Summary

Page 4: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 4/74

15744 - Fall 2004 Lecture 3 4

Routing Hierarchies

• Flat routing doesn’t scale  – Each node cannot be expected to have routes

to every destination (or destination network)

• Key observation

 – Need less information with increasing distanceto destination

• Two radically different approaches for

routing – The area hierarchy?

 – The landmark hierarchy (discuss in routingalternatives)?

Page 5: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 5/74

15744 - Fall 2004 Lecture 3 5

 Areas

• Divide network into areas –  Areas can have nested

sub-areas

 – Constraint: no pathbetween two sub-areas of

an area can exit that area• Hierarchically address

nodes in a network – Sequentially number top-

level areas

 – Sub-areas of area arelabeled relative to that area

 – Nodes are numberedrelative to the smallestcontaining area

1 2

3

1.1

1.2

2.1  2.2

3.1  3.2

2.2.1

2.2.2

1.2.1

1.2.2

Page 6: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 6/74

15744 - Fall 2004 Lecture 3 6

Routing

• Within area

 – Each node has routes to every other node

• Outside area – Each node has routes for other top-level

areas only

 – Inter-area packets are routed to nearest

appropriate border router

• Can result in sub-optimal paths 

Page 7: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 7/74

15744 - Fall 2004 Lecture 3 7

Path Sub-optimality

1 2

3

1.1

1.2

2.1 2.2

3.1 3.2

2.2.1

3 hop red path

vs.

2 hop green path

start

end

3.2.1 

1.2.1 

Page 8: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 8/74

15744 - Fall 2004 Lecture 3 8

 A Logical View of the Internet

Tier 1 Tier 1

Tier 2Tier 2

Tier 2

Tier 3

• National (Tier 1 ISP) – “Default-free” with

global reachability info

Eg: AT & T, UUNET,

Sprint• Regional (Tier 2 ISP)

 – Regional or country-

wide

Eg: Pacific Bell• Local (Tier 3 ISP)

Eg: Telerama DSL

Customer

Provider

Page 9: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 9/74

15744 - Fall 2004 Lecture 3 9

Landmark Routing: Basic Idea

Source wants to reach LM0[a], whose address is c.b.a: •Source can see LM2[c], so sends packet towards c

•Entering LM1[b] area, first router diverts packet to b

•Entering LM0[a] area, packet delivered to a

- Not shortest path

- Packet does not necessarily

follow specified landmarks

- No policy routing

- Not source routing

-Why

-Small routing tables-Dynamic, self

configuring algorithms

Page 10: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 10/74

15744 - Fall 2004 Lecture 3 10

Outline

• Need for hierarchical routing

• BGP –  ASes, Policies

 – BGP Attributes

 – BGP Path Selection – iBGP

 – Inferring AS relationships

• Problems with BGP – Convergence

 – Sub optimal routing

• Overlay Routing and Multihoming

• Summary

Page 11: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 11/74

15744 - Fall 2004 Lecture 3 11

 Autonomous Systems (ASes)

•  Autonomous Routing Domain – Glued together by a common administration, policies etc

•  Autonomous system – is a specific case of an ARD –  ARD is a concept vs AS is an actual entity that participates in

routing

 – Has an unique 16 bit ASN assigned to it and typicallyparticipates in inter-domain routing

• Examples: – MIT: 3, CMU: 9

 – AT&T: 7018, 6341, 5074, …

 – UUNET: 701, 702, 284, 12199, …  – Sprint: 1239, 1240, 6211, 6242, … 

• How do ASes interconnect to provide global connectivity

• How does routing information get exchanged

Page 12: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 12/74

15744 - Fall 2004 Lecture 3 12

Nontransit vs. Transit ASes

ISP 1

ISP 2

Nontransit AS

might be a corporateor campus network.Could be a “content

provider” 

NET ATraffic NEVERflows from ISP 1through NET A to ISP 2(At least not intentionally!)

IP traffic

Page 13: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 13/74

15744 - Fall 2004 Lecture 3 13

Customers and Providers

Customer pays provider for access to the Internet

provider

customer

IP trafficprovider customer

Page 14: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 14/74

15744 - Fall 2004 Lecture 3 14

The Peering Relationship

peer peer

customerprovider

Peers provide transit between

their respective customers

Peers do not provide transitbetween peers

Peers (often) do not exchange $$$trafficallowed

traffic NOTallowed

 A

B

C

Page 15: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 15/74

15744 - Fall 2004 Lecture 3 15

Peering Wars

• Reduces upstream transitcosts

• Can increase end-to-endperformance

• May be the only way toconnect your customersto some part of theInternet (“Tier 1”)

• You would rather have

customers

• Peers are usually your

competition

• Peering relationships

may require periodic

renegotiation

Peering struggles are by far the mostcontentious issues in the ISP world!

Peering agreements are often confidential.

Peer Don’t Peer  

Page 16: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 16/74

15744 - Fall 2004 Lecture 3 16

Routing in the Internet

• Link state or distance vector?

 – No universal metric – policy decisions

• Problems with distance-vector:

 – Bellman-Ford algorithm may not converge

• Problems with link state:

 – Metric used by routers not the same – loops

 – LS database too large – entire Internet

 – May expose policies to other AS’s 

Page 17: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 17/74

15744 - Fall 2004 Lecture 3 17

Solution: Distance Vector with

Path• Each routing update carries the entire path

• Loops are detected as follows:

 – When AS gets route check if AS already in

path

• If yes, reject route

• If no, add self and (possibly) advertise route further

•  Advantage: – Metrics are local - AS chooses path, protocol

ensures no loops

Page 18: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 18/74

15744 - Fall 2004 Lecture 3 18

BGP-4• BGP = Border Gateway Protocol• Is a Policy-Based routing protocol

• Is the EGP of today’s global Internet 

• Relatively simple protocol, but configuration is complex and the

entire world can see, and be impacted by, your mistakes.

• 1989 : BGP-1 [RFC 1105]

 –  Replacement for EGP (1984, RFC 904) 

• 1990 : BGP-2 [RFC 1163]

• 1991 : BGP-3 [RFC 1267]

• 1995 : BGP-4 [RFC 1771]

 –  Support for Classless Interdomain Routing (CIDR) 

Page 19: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 19/74

15744 - Fall 2004 Lecture 3 19

BGP Operations (Simplified) 

Establish session onTCP port 179

Exchange all

active routes

Exchange incremental

updates

AS1

AS2

While connection

is ALIVE exchange

route UPDATE messages

BGP session

Page 20: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 20/74

15744 - Fall 2004 Lecture 3 20

Interconnecting BGP Peers

• BGP uses TCP to connect peers

•  Advantages: – Simplifies BGP

 – No need for periodic refresh - routes are valid untilwithdrawn, or the connection is lost

 – Incremental updates

• Disadvantages

 – Congestion control on a routing protocol? – Inherits TCP vulnerabilities!

 – Poor interaction during high load

Page 21: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 21/74

15744 - Fall 2004 Lecture 3 21

Four Types of BGP Messages

• Open : Establish a peering session.• Keep Alive : Handshake at regular intervals.

• Notification : Shuts down a peering session.

• Update : Announcing new routes or withdrawing

previously announced routes.

announcement=

prefix + attributes values

Page 22: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 22/74

15744 - Fall 2004 Lecture 3 22

Policy with BGP

• BGP provides capability for enforcing various policies

• Policies are not part of BGP: they are provided to BGPas configuration information

• BGP enforces policies by choosing paths from multiple

alternatives and controlling advertisement to other AS’s • Import policy

 – What to do with routes learned from neighbors?

 – Selecting best path

• Export policy – What routes to announce to neighbors?

 – Depends on relationship with neighbor

Page 23: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 23/74

15744 - Fall 2004 Lecture 3 23

Examples of BGP Policies

•  A multi-homed AS refuses to act as transit

 – Limit path advertisement

•  A multi-homed AS can become transit for

some AS’s  – Only advertise paths to some AS’s 

 – Eg: A Tier-2 provider multi-homed to Tier-1providers

• An AS can favor or disfavor certain AS’sfor traffic transit from itself

Page 24: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 24/74

15744 - Fall 2004 Lecture 3 24

Export Policy

•  An AS exports only best paths to its neighbors – Guarantees that once the route is announced the AS

is willing to transit traffic on that route

• To Customers

 –  Announce all routes learned from peers, providersand customers, and self-origin routes

• To Providers –  Announce routes learned from customers and self-

origin routes

• To Peers –  Announce routes learned from customers and self-

origin routes

Page 25: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 25/74

15744 - Fall 2004 Lecture 3 25

Import Routes

From

peer

From

peer

From

provider

From

provider

From

customerFrom

customer

provider route customer routepeer route ISP route

Page 26: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 26/74

15744 - Fall 2004 Lecture 3 26

Export Routes

To

peer

To

peer

To

customer

To

customer

To

provider

From

provider

provider route customer routepeer route ISP route

filters

block

Page 27: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 27/74

15744 - Fall 2004 Lecture 3 27

BGP Route Processing

Best Route

Selection

Apply Import

Policies

Best Route

Table

Apply Export

Policies

Install forwarding

Entries for best

Routes.

Receive

BGP

Updates

Best

Routes

Transmit

BGP

Updates

 Apply Policy =

filter routes &

tweak attributes

Based on

 Attribute

Values

IP Forwarding Table

 Apply Policy =

filter routes &

tweak attributes

Open ended programming.

Constrained only by vendor configuration language

Page 28: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 28/74

15744 - Fall 2004 Lecture 3 28

BGP UPDATE Message

• List of withdrawn routes

• Network layer reachability information

 – List of reachable prefixes

• Path attributes

 – Origin

 – Path

 – Metrics

•  All prefixes advertised in message have

same path attributes

Page 29: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 29/74

15744 - Fall 2004 Lecture 3 29

Path Selection Criteria

• Information based on path attributes

•  Attributes + external (policy) information

• Examples: – Hop count

 – Policy considerations

• Preference for AS

• Presence or absence of certain AS

 – Path origin

 – Link dynamics

Page 30: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 30/74

15744 - Fall 2004 Lecture 3 30

Important BGP Attributes

• Local Preference

•  AS-Path

• MED• Next hop

Page 31: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 31/74

15744 - Fall 2004 Lecture 3 31

LOCAL PREF

• Local (within an AS) mechanism to provide relative

priority among BGP routers

R1 R2

R3 R4I-BGP

 AS 256

 AS 300

Local Pref = 500 Local Pref =800

 AS 100

R5

 AS 200

Page 32: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 32/74

15744 - Fall 2004 Lecture 3 32

LOCAL PREF – Common Uses

• Handle routes advertised to multi-homed

transit customers

 – Should use direct connection (multihoming

typically has a primary/backup arrangement)

• Peering vs. transit

 – Prefer to use peering connection, why?

• In general, customer > peer > provider

 – Use LOCAL PREF to ensure this

Page 33: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 33/74

15744 - Fall 2004 Lecture 3 33

 AS_PATH

• List of traversed AS’s 

• Useful for loop checking and for path-based route selection (length, regexp)

 AS 500

 AS 300

 AS 200 AS 100

180.10.0.0/16 300 200 100

170.10.0.0/16 300 200 

170.10.0.0/16 180.10.0.0/16

Page 34: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 34/74

15744 - Fall 2004 Lecture 3 34

Multi-Exit Discriminator (MED)

• Hint to external neighbors about the

preferred path into an AS

 – Non-transitive attribute

 – Different AS choose different scales

• Used when two AS’s connect to each

other in more than one place

Page 35: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 35/74

15744 - Fall 2004 Lecture 3 35

MED

• Typically used when two ASes peer at multiple locations

• Hint to R1 to use R3 over R4 link

• Cannot compare AS40’s values to AS30’s 

R1 R2

R3 R4

 AS 30

 AS 40

180.10.0.0

MED = 120180.10.0.0

MED = 200

 AS 10

180.10.0.0

MED = 50

Page 36: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 36/74

15744 - Fall 2004 Lecture 3 36

MED• MED is typically used in provider/subscriber scenarios

• It can lead to unfairness if used between ISP because it

may force one ISP to carry more traffic:

SF

NY

• ISP1 ignores MED from ISP2

• ISP2 obeys MED from ISP1• ISP2 ends up carrying traffic most of the way

ISP1

ISP2

Page 37: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 37/74

15744 - Fall 2004 Lecture 3 37

Other Attributes

• ORIGIN

 – Source of route (IGP, EGP, other)

• NEXT_HOP

 – Address of next hop router to use

• Check out http://www.cisco.com for full

explanation

• Question: Too many choices/ attributes

how to select routes !

Page 38: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 38/74

15744 - Fall 2004 Lecture 3 38

Route Selection Process

Highest Local Preference

Shortest ASPATHLowest MED

i BGP < e BGP

Lowest IGP cost

to BGP egress

Lowest router ID

traffic engineering

Enforce relationships

Throw up hands and

break ties

Page 39: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 39/74

15744 - Fall 2004 Lecture 3 39

Internal vs. External BGP

R3 R4

R1

R2

E-BGP

•BGP can be used by R3 and R4 to learn routes

•How do R1 and R2 learn routes?

•Option 1: Inject routes in IGP

•Only works for small routing tables

•Option 2: Use I-BGP

 AS1  AS2

Page 40: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 40/74

15744 - Fall 2004 Lecture 3 40

Internal BGP (I-BGP)

• Same messages as E-BGP

• Different rules about re-advertising

prefixes:

 – Prefix learned from E-BGP can be advertised

to I-BGP neighbor and vice-versa, but

 – Prefix learned from one I-BGP neighbor

cannot be advertised to another I-BGPneighbor

 – Reason: no AS PATH within the same AS and

thus danger of looping.

Page 41: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 41/74

15744 - Fall 2004 Lecture 3 41

Internal BGP (I-BGP)

R3 R4

R1

R2

E-BGP

I-BGP

• R3 can tell R1 and R2 prefixes from R4

• R3 can tell R4 prefixes from R1 and R2• R3 cannot tell R2 prefixes from R1

R2 can only find these prefixes through a direct connection to R1

Result: I-BGP routers must be fully connected (via TCP)!

• contrast with E-BGP sessions that map to physical links 

 AS1  AS2

R t R fl t

Page 42: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 42/74

15744 - Fall 2004 Lecture 3 42

Route Reflector

eBGP update

iBGP updates

Mesh does not scale

RR RR

RR

Each RR passes only best routes, no longer

N^2 scaling problem

Page 43: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 43/74

15744 - Fall 2004 Lecture 3 43

Policy Impact

• Different relationships – Transit, Peering

• Export policies selective export

• “Valley-free” routing  – Number links as (+1, 0, -1) for customer-to-

provider, peer and provider-to-customer

 – In any path should only see sequence of +1,

followed by at most one 0, followed bysequence of -1

Page 44: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 44/74

15744 - Fall 2004 Lecture 3 44

How to infer AS relationships?

• Can we infer relationship from the AS graph – From routing information

 – From size of ASes /AS topology graph

 – From multiple views and route announcements

• [Gao01] – Three-pass heuristic

 – Data from University of Oregon RouteViews

• [SARK01]

 – Data from multiple vantage points – Formulate TOR problem

 – Heuristic for solving the relationship assignment

Page 45: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 45/74

15744 - Fall 2004 Lecture 3 45

[Gao00] Basic Algorithm

• Phase 1: Identify the degrees of the ASes fromthe tables

• Phase 2: Annotate edges with “transit” relation  –  AS u transits traffic for AS v if it provides its

provider/peer routes to v.

• Phase 3: Identify P2C, C2P, Sibling edges – P2C -> If and only if u transits for v, and v does not,

Sibling otherwise

 – Peering relationship ?

• Refined Algorithm : Another parameter L

Page 46: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 46/74

15744 - Fall 2004 Lecture 3 46

How does Phase 2 work?

• Notion of Valley free routing

 – Each AS path can be

• Uphill

• Downhill

• Uphill – Downhill

• Uphill – P2P

• P2P -- Downhill

• Uphill – P2P – Downhill

• How to identify Uphill/Downhill

 – Heuristic: Identify the highest degree AS to be the

end of the uphill path (path starts from source)

Page 47: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 47/74

15744 - Fall 2004 Lecture 3 47

Observations from [Gao00]

• Heuristic to identify top provider does not work

•  Algorithm inferences verified from sources within AT & T.

• Majority are P2C, few peering, few sibling – Peering is few because of the dataset used?

 – Sibling relationships are becoming more common• Mergers, takeovers, backup relationships

•  AS relationships are often complex – inferredrelationships are “dominant commercialrelationships” 

Page 48: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 48/74

15744 - Fall 2004 Lecture 3 48

Questions..

• Is inter-AS relationship – Prefix based

 – Customer based

 – Independent?

• Is the degree based heuristic valid?

•  Are peering relationships underestimated?

• How useful is inferring the relationships

 – Policy violations –  Anomaly detection?

 – Is this information revealed against the providers will?

Page 49: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 49/74

15744 - Fall 2004 Lecture 3 49

BGP Inefficiencies,Overlays and

Multihoming

 Aditya Akella

Page 50: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 50/74

15744 - Fall 2004 Lecture 3 50

BGP Complexity

• BGP is a very complicated protocol – Too many knobs

 – Need to accommodate (sub-optimal)ISP policies

 – Requires complex, humanconfiguration

• For all its complexity, BGP offersno guarantees – Performance??

 – Reliability?? – Correctness??

 – Reachability??

• All of BGPs complexity begets… 

Headache!

Page 51: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 51/74

15744 - Fall 2004 Lecture 3 51

BGP Pitfalls and Problems

• Pitfalls and problems

 – Misconfiguration

 – Convergence

 – Performance

 – Reliability

 – Stability

 – Security

 – And the list goes on… 

Page 52: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 52/74

15744 - Fall 2004 Lecture 3 52

Favorite Scapegoat!

BGP

Networkingcommunity

Page 53: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 53/74

15744 - Fall 2004 Lecture 3 53

Misconfiguration[Mahajan02Sigcomm]

• Origin misconfiguration: accidentally inject

routes for prefixes into global BGP tables

Old Route New Route

Self deaggregation(failure to

aggregate)

a.b.0.0/16 X Y Z a.b.c.0/24 X Y Z

Related origin

(likely connected to

the network –

 human error)

a.b.0.0/16 X Y Z a.b.0.0/16 X Y

a.b.0.0/16 X Y Z O

a.b.c.0/24 X Y

a.b.c.0/24 X Y Z O

Foreign origin

(address space

hijack!)

a.b.0.0/16 X Y Z a.b.0.0/16 X Y O

a.b.c.0/24 X Y O

e.f.g.h/i X Y O

Page 54: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 54/74

15744 - Fall 2004 Lecture 3 54

Misconfiguration

• Export misconfiguration: export route to a peerin violation of policy

Export Policy Violation

Provider AS Provider Route exported to provider wasimported from a provider

Provider AS Peer Route exported to peer was

imported from a provider

Peer

 AS

 Provider Route exported to provider wasimported from a peer

Peer AS Peer Route exported to peer was

imported from a peer

Page 55: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 55/74

15744 - Fall 2004 Lecture 3 55

Interesting Observations

• Origin misconfig – 72% of new routes may be misconfig

 – 11-13% of misconfig incidents affect connectivity• Pings and e-mail checks

 – Self de-aggregation is the main cause

• Export misconfig – Upto 500 misconfiguration incidents per day

 –  All forms are prevalent, although provider-AS-provideris more likely

Page 56: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 56/74

15744 - Fall 2004 Lecture 3 56

Effects and Causes

• Effects – Routing load

 – Connectivity disruption

 – Extra traffic

 – Policy violation

• Causes (Origin misconfig) – Router vendor software bugs:

announce and withdraw routeson reboot

 – Reliance on upstream filtering

 – New configuration not saved tostable storage (separatecommand and no autosave!)

 – Hijacks of address spaces

 – Forgotten to install filter

 – Human operators and poorinterface

P1 P2

A

C

• Intended policy: Provide transit to Cthrough link A-C

• Configured policy: Export all routes

originated by C to P1 and P2

• Correct policy: export only when AS

path is “C” 

Export Misconfig

Page 57: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 57/74

15744 - Fall 2004 Lecture 3 57

BGP Convergence[Labovitz00Sigcomm]

• Conventional beliefs – Path vector converges faster than traditional DV(eliminates the count to infinity problem)

 – Internet path restoration takes order of 10s of seconds

• Convergence

 – Recovery after a fault may take as much as ten minutes – Single routing fault could result in multiple

announcements and withdrawals

 – Loss and RTT around times of faults are much worse

• Upon route withdrawal, explore paths of increasing

length – In the worst case, could explore n! paths

 – Depends which messages are processed and when

• Limit between update message could reducemessages

 – Forces all outstanding messages to be processed

IBGP P bl

Page 58: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 58/74

15744 - Fall 2004 Lecture 3 58

IBGP Problems[Griffin02Sigcomm]

• Route reflectors could impose signaling

and forwarding anamolies instability!

Ri is a reflector for Ci (updates sent between Ri and Ci, i=1, 2)

Ri is a BGP

router

announcingPi into the

network

Ci will only

know about Pi 

and it as bestpath

But Ci---Pi shortest path is Ci Ci+1Ri and this causes a forwarding loop!

Page 59: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 59/74

15744 - Fall 2004 Lecture 3 59

End-to-End Routing Behavior[Paxson96Sigcomm]

• Large scale routing behavior as seen by end-hosts, based on analysis of traceroutes

• Pathologies: persistent routing loops, routingfailures and long connectivity outages

• Stability: 9% or routes changed every 10s of

minutes, 30% about ~6hrs and 68% took a fewdays

• Symmetry: more than half of paths probed were

asymmetric at router level

Inefficiencies in BGP &

Page 60: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 60/74

15744 - Fall 2004 Lecture 3 60

Inefficiencies in BGP &

Internet Routing

• Route convergence and oscillations

• Poor reliability

 – No way to exploit redundancy in Internet

paths• Inefficiency: sub-optimal RTTs and

throughputs

 – What are some of the causes?• Policies in routing: Inter-domain and Intra-domain

• Lack of direct routes, “sparseness” of the Internetgraph

I ffi i f R t

Page 61: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 61/74

15744 - Fall 2004 Lecture 3 61

Inefficiency of Routes[Spring03Sigcomm]

• Three classes of reasons for poor performance(“inflation”)  – Intra-domain topology and policy

• Topology: no direct link between all cities

• Routing policy: “shortest paths” may be avoided due toengineering

 – ISP Peering• Peeering topology: limited peering between ISPs

• Peering policy: hot-potato routing or early-exit routing

 – Inter-domain• Topology: AS graph is sparse

• Inter-domain policies: policies are policies

Page 62: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 62/74

15744 - Fall 2004 Lecture 3 62

Path Inflation Summary

Performance: End to End

Page 63: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 63/74

15744 - Fall 2004 Lecture 3 63

Performance: End-to-End

Perspective

• From an end-to-end view… 

 – Is there a way of extracting better

performance?

• Is there scope?• How do we realize this?

• Scope: Savage99, Akella03, Akella04• Reality: UW’s “Detour” system, MIT’s

RON, Akamai’s SureRoute, CMU’s Route

Control

Q tif i P f L

Page 64: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 64/74

15744 - Fall 2004 Lecture 3 64

Quantifying Performance Loss[Savage99Sigcomm]

• Measure round trip time (RTT) and loss

rate between pairs of hosts

•  Alternate path characteristics

 – 30-55% of hosts had lower latency

 – 10% of alternate routes have 50% lower

latency

 – 75-85% have lower loss rates

Page 65: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 65/74

15744 - Fall 2004 Lecture 3 65

Bandwidth Estimation

• RTT & loss for multi-hop path – RTT by addition

 – Loss either worst or combine of hops – why?

• Large number of flows combination ofprobabilities

• Small number of flows worst hop

• Bandwidth calculation

 – TCP bandwidth is based primarily on loss andRTT

• 70-80% paths have better bandwidth

• 10-20% of paths have 3x improvement

Possible Sources of Alternate

Page 66: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 66/74

15744 - Fall 2004 Lecture 3 66

Possible Sources of Alternate

Paths• A few really good or bad AS’s

 – No, benefit of top ten hosts not great

• Better congestion or better propagation

delay?

 – How to measure?

• Propagation = 5th percentile of delays

 – Both contribute to improvement ofperformance

Page 67: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 67/74

15744 - Fall 2004 Lecture 3 67

Overlay Networks

• Basic idea:

 – Treat multiple hops through IP network as one hop in overlay network

 – Run routing protocol on overlay nodes

• Why?

 – For performance – like the Savage 99 paper showed

 – For efficiency – can make core routers very simple

• E.g. CSFQ,

•  Also aid deployment. E.g. Active networks

 – For functionality – can provide new features such as multicast, active

processing

Page 68: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 68/74

15744 - Fall 2004 Lecture 3 68

Future of Overlay

•  Application specific overlays

 – Why should overlay nodes only do routing?

• Caching

 – Intercept requests and create responses

• Transcoding

 – Changing content of packets to match

available bandwidth

• Peer-to-peer applications

Page 69: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 69/74

15744 - Fall 2004 Lecture 3 69

Overlay Challenges

• “Routers” no longer have complete knowledge about link they areresponsible for

• How do you build efficient overlay – Probably don’t want all N2 links – which links to create?

 – Without direct knowledge of underlying topology how to know what’snearby and what is efficient?

• Do we need overlays for performance?

Page 70: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 70/74

15744 - Fall 2004 Lecture 3 70

Number of Route Choices

• Flexible control of end-

to-end pathmany route choices

Multiple candidatepaths

Single path

Multiple BGP

paths

• BGP: one path via each

ISPchoices linked to #ISPs

Few more routechoices…? 

Page 71: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 71/74

15744 - Fall 2004 Lecture 3 71

Route Selection Mechanism

• BGP: simple, coarsemetrics such as least AShops, policy

Best performingpath

Least AS hopsPolicy compliant

Current best

performing

BGP path

• Overlays: complex,performance-orientedselection

Sophisticated selection among

multiple BGP routes

Smart

selection

“Multihoming route control” 

O l R ti M ltih i R t C t l

Page 72: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 72/74

15744 - Fall 2004 Lecture 3 72

Overlay Routing vs Multihoming Route Control[Akella04Sigcomm]

>

>

~

~

1-multihoming

k-multihoming

k-multihoming

1-overlays

1-overlays

k-overlays

~

Page 73: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 73/74

15744 - Fall 2004 Lecture 3 73

Overlay Routing vs. Multihoming Route Control

Cost

Operational

issues

Route Control Overlay Routing

Sprint$$

Genuity$$

ATT

$$

Overlay provider$$

ATT

$$

Overlay node

forces inter-mediate ISPto providetransit

/18 netblock

Announce/20 sub-blocksto ISPs

If all multihomedends do this

Routing table expansion Bad interactions with policies

Connectivity fees Connectivity fees + overlay fee

Page 74: Inter-Domain Routing Lecture

8/13/2019 Inter-Domain Routing Lecture

http://slidepdf.com/reader/full/inter-domain-routing-lecture 74/74

Summary

• Route control similar to overlay routing for most practical

purposes

• Overlays very useful for deploying functionality

 – Multicast, VPNs, QoS, security

• But overlays may be overrated for end-to-end

performance and resilience

• Don’t abandon BGP – there’s still hope of extracting good

performance and availability