Internet Routing: Measurement, Modeling, and Analysis

Dr. Jia Wangjiawang@research.att.com

AT&T Labs ResearchFlorham Park, NJ 07932, USA

http://www.research.att.com/~jiawang/

Prof. Zhuoqing Morley Maozmao@umich.eduDepartment of EECS

University of Michigan Ann Arbor, MI 48109, USA

http://www.eecs.umich.edu/~zmao/

ACM Sigmetrics 2005 Tutorial

2

Outline

1. Overview of Inter-domain routing2. Measuring inter-domain paths3. BGP Measurement4. BGP Modeling

Our opinions should not be taken to represent AT&T policies

Part I: Overview of Inter-domain Routing

4

Internet

Loose cooperative effort of Internet Service Providers (ISPs) E.g., AT&T, Sprint, UUNet, AOL

Best effort service Connectedness

Anyone connected to the Internet can exchange traffic with anyone else connected to the Internet

5

: Routing session

routes Control plane:exchange routes

Internet routing

rusty.cs.berkeley.eduIP=169.229.62.116

Prefix=169.229.0.0/16

www.cnn.comIP=64.236.16.52

Prefix=64.236.16.0/20

Internet

IP traffic

Data plane:forward traffic

Fail over to alternate route

6

Internet routing domain

Autonomous routing domain Network devices under same technical and

administrative control Common routing policy E.g., ISPs, enterprise networks

Autonomous system Autonomous routing domain with an AS number (ASN) AS numbers: 16 bits integer

Public AS number: 1 – 64511 Private AS number: 64512 – 65535 Examples

AT&T: 7018, 6431, … Sprint: 1239, 1240, … MIT: 3

7

More than 20,000 ASes today

Berkeley

Internet

CNN

Calren

Level3

GNN

IP traffic

Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

AutonomousSystem

Berkeley

Calren

Level3 Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

Berkeley

Calren

Level3 Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

8

Internet routing architecture

IP traffic

Berkeley CNN

Level3

Internet

Calren GNN

Inter-domain routing

Intra-domain routing

9

Intra-domain routing

Run within a certain network infrastructure Optimize routes taken between points

within a network Internal Gateway Protocols (IGPs)

Metrics based OSPF (Open Shortest Path First) RIP (Routing Information Protocol) IS-IS (Intermediate System to Intermediate

System)

10

Inter-domain routing

Run between networks Provide full connectivity of entire

Internet External Gateway Protocol (EGP)

Policy based BGP (Border Gateway Protocol)

11

Link state protocols

Examples: OSPF, IS-IS Based on Dijkstra’s shortest path

computation Each router periodically floods immediate

reachability information to other routers Fast convergence High communication and computation

overhead Not scalable for large networks

Requires periodic refreshes

12

Vectoring protocols

Distance vs. Path Vector Distance: hop count (RIP) Path: entire path (BGP)

Helps identify loops Supports policy-based routing based on path

Minimal communication overhead Takes longer to converge, i.e., in

proportion to the maximum path length

13

Link state vs. vectoring

OSPFIS-IS

RIP

BGP

IGP

EGP

Link state Vectoring

BGP is a path vector protocol

14

Classful addressing

IPv4: 32 bits Five classes of networks

Class

Address

Mask # of networks

# of hosts

A 0* 255.0.0.0 128 ~1.6M

B 10* 255.255.0.0 16384 65535

C 110* 255.255.255.0

~2.1M 255

D Used for multicast

E Reserved and currently unusedImprove scaling factor of routing in the Internet => classless

15

CIDR: Classless Inter-domain Routing (RFC1519)

No implicit mask based on the class of the network

Explicit masks passed in the routing protocol Allow aggregation and hierarchical routing

00001100 00100110 00000000 00000000

11111111 11111111 11000000 00000000

IP address: 12.70.0.0 Mask: 255.255.252.0

CIDR representation: 12.70.0.0/22

Address

Mask

Network prefixHost

identifier

00001100 00100110 00000000 00000000

11111111 11111111 11000000 00000000

16

Address aggregation

Internet

12.70.1.0/24

12.70.2.0/24

12.70.3.0/2412.70.0.0/24

ISP AISP B

12.70.0.0/2212.71.0.0/16

12.71.0.0/16

17

Routing and forwarding

Routing The decision process of choosing

optimal path that is consistent with the administrative or technical policy

Forwarding The act of receiving a packet, doing a

lookup, and copying a packet to the next hop

18

Classless forwarding

Internet

135.120.0.1

12.70.0.20

IP traffic

Prefix Next hop12.70.0.0/24 10.20.0.112.70.0.0/16 10.20.1.112.0.0.0/8 10.20.128.10.0.0.0 10.20.128.10

10.20.0.1

10.20.1.1

10.20.128.1

10.20.128.10

19

Inter-domain routing with CIDR support

BGP-4 [RFC1771] De facto EGP Carry routing information between ASes Path vector protocol Policy based routing Run on top of TCP for reliability Basic operations

Set up BGP session Exchange all candidate routes Send incremental updates

20

Establish BGP session

12.10.0.1 12.10.0.2

Establish neighboring session between 12.10.0.1 and 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

TCP 179

21

Exchange all candidate routes

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

22

Send incremental updates

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Withdraw 12.9.0.0/16

23

BGP messages

OPEN: set up a peering session UPDATE: announce new routes or

withdraw previously announced routes

NOTIFICATION: shut down a peering session

KEEPALIVE: confirm active connection at regular interval

24

Internal vs. external BGP

Internet I-BGP

E-BGP

AS A

AS B

AS C

E-BGPupdate

I-BGPupdate

I-BG

Pup

date

25

Scaling I-BGP for large AS

Route reflectors Confederations

E-BGP update

RR RR

Only best paths being sent by RR

AS 1000

EBGP

EB

GP

EBGPIBGP IBGP

AS 65010 AS 65020

26

Establish connectivity

135.120.0.0/16

12.10.0.1

12.10.0.2

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.5 2 1

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

27

IGP and BGP working together

135.120.0.0/1612.10.0.1

12.10.0.2

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

10.10.0.1

Prefix Next hop12.10.0.0/30 10.10.0.1135.120.0.0/16 10.10.0.1

12.10.0.0/30

28

Policy routing

ISP1

ISP4ISP3

Cust1 Cust2

ISP2

traffic

traffic

Connectivity DOES NOT imply reachability!

Policy determines how traffic can flow on the Internet

29

BGP routing process

Applyinputpolicy

Routesreceived from peers

Selectbest route

Bestroutes

Applyoutputpolicy

Routes advised to peers

Routingtable

Forwardingtable

BGP is not shortest path routing!

30

Best route selection

Highest local preference Shortest AS path Lowest MED (Multi-Exit-

Discriminator) I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress Tie breaking rules

31

Best route selection

Highest local preference To enforce economical relationships

between domains Shortest AS path Lowest MED (Multi-Exit-Discriminator) I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress Tie breaking rules

32

Best route selection

Highest local preference Shortest AS path

Compare the quality of routes, assuming shorter AS-path length is better

Lowest MED (Multi-Exit-Discriminator) I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress Tie breaking rules

33

Best route selection

Highest local preference Shortest AS path Lowest MED (Multi-Exit-Discriminator)

To implement “cold potato” routing between neighboring domains

I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress Tie breaking rules

34

Best route selection

Highest local preference Shortest AS path Lowest MED (Multi-Exit-Discriminator) I-BGP < E-BGP

Prefer EBGP routes to IBGP routes Lowest I-BGP cost to E-BGP egress Tie breaking rules

35

Best route selection

Highest local preference Shortest AS path Lowest MED (Multi-Exit-Discriminator) I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress

Prefer routes via the nearest IGP neighbor To implement “hot potato” routing

Tie breaking rules

36

Best route selection

Highest local preference Shortest AS path Lowest MED (Multi-Exit-Discriminator) I-BGP < E-BGP Lowest I-BGP cost to E-BGP egress Tie breaking rules

Router ID based: lowest router ID Age based: oldest route

37

BGP route propagation

Not all possible routes propagate Commercial relationships determine

policies for Route import Route selection Route export

38

Typical AS relationships

Provider-customer customer pay money for transit

Peer-peer typically exchange respective customers’

traffic for free

Siblings Mutual transit agreement Provide connectivity to the rest of the

Internet for each other

39

AS relationships translate into BGP export rules

Export to a provider or a peer Allowed: its routes and routes of its

customers and siblings Disallowed: routes learned from other

providers or peers Export to a customer or a sibling

Allowed: its routes, the routes of its customers and siblings, and routes learned from its providers and peers

40

Which AS paths are legal?

Valley-free: After traversing a provider-customer

or peer-peer edge, cannot traverse a customer-provider or peer-peer edge

Invalid path: >= 2 peer links, downhill-uphill, downhill-peer, peer-uphill

41

Example of valley-free paths

XX

[1 2 3], [1 2 6 3] are valley-free

[1 4 3], [1 4 5 3] are not valley free

42

Inferring AS relationships

Identify the AS-level hierarchy of Internet Not shortest path routing

Predict AS-level paths Traffic engineering Understand the Internet better Correlate with and interpret BGP update Identify BGP misconfigurations

E.g., errors in BGP export rules

43

Existing approaches

On inferring Autonomous Systems Relationships in the Internet, by L. Gao, IEEE Global Internet, 2000.

Characterizing the Internet hierarchy from multiple vantage points, by L. Subramanian, S. Agarwal, J. Rexford, and R. Katz, IEEE Infocom, 2002.

Computing the Types of the Relationships between Autonomous Systems, by G. Battista, M. Patrignani, and M. Pizzonia, IEEE Infocom, 2003.

On AS-level Path Inference, by Z. Mao, L. Qiu, J. Wang, and Y. Zhang, ACM Sigmetrics, 2005.

44

Policy routing causes path inflation

End-to-end paths are significantly longer than necessary

Why? Topology and routing policy choices within

an ISP, between pairs of ISPs, and across the global Internet

Peering policies and interdomain routing lead to significant inflation

Interdomain path inflation is due to lack of BGP policy to provide convenient engineering of good paths across ISPs

45

Path inflation

Based on [Mahajan03]

Comparing actual Internet paths with hypothetical “direct” link