CONFIDENTIAL

Multihoming using Juniper MX80

Howard Hart

A short pause for a commercial

Rated #1

$ Free Phone

Service

Break through

Voice Quality

Unique New

Voice Services

....

Platform for Future

Managed Services

Ooma Business

• Hosted VoIP services used by hundreds of

thousands of customers

• Carrying over a hundred million minutes of

call traffic per month

• Pushing millions of packets per second

• Standards based VoIP using Session

Initiation Protocol (SIP) signaling and Real

Time Protocol (RTP) media

Ooma Challenges

• RTP is unforgiving and so are customers

– One dropped packet and user hears a pop

• Can buffer out-of-order RTP packets up to

100 msecs.

– Just five 20 msec packets for most codecs

• More than one network switch

hasn’t been up to the challenge

at our scale

UNCLE

?

Why Juniper?

• We’ve got enough challenges--don’t need to add router limitations to the mix

• MX80 provides wirespeed packet-switching with zero buffering

– Average 40 msecs coast to coast

– Plus buffer bloat delays

– Plus local head-of-line blocking on NAT

• 50 million pps capacity on MX80 gives us lots of headroom

Why multihome?

• Carriers go down

• Hardware fails

• Maybe some of your destinations are

cheaper through an alternate carrier

• Your resume’s a little light and you’re not

up to speed yet on Web 2.1

Topology

• Control inbound flow via BGP

announcements and AS prepending

• Control outbound flow via VRRP

• Automate failover using Juniper interface

tracking feature

Network Diagram

172.16.1.0/30 2

1

2

Interwebs

ASN 1000

192.168.1.0/24

ASN 12345

10.1.1.0/30 2

1

xe-0/0/0

xe-0/0/1 xe-0/0/2

Joesgarage

ASN 2000

172.16.1.4/30 6

5

3

Interwebs

10.1.1.4/30 6

5

xe-0/0/0

xe-0/0/1 xe-0/0/2

Joesgarage

Router 1 Router 2

VRRP Configuration

Router1:

set interfaces xe-0/0/0 unit 0 family inet address 192.168.1.2/24 vrrp-group 0

edit interfaces xe-0/0/0 unit 0 family inet address 192.168.1.2/24 vrrp-group 0

set virtual-address 192.168.1.1;

set priority 100;

set track interface xe-0/0/1 priority-cost 10;

set track interface xe-0/0/2 priority-cost 5;

Router2:

set interfaces xe-0/0/0 unit 0 family inet address 192.168.1.3/24 vrrp-group 0

edit interfaces xe-0/0/0 unit 0 address 192.168.1.3/24 vrrp-group 0

set virtual-address 192.168.1.1;

set priority 120;

set track interface xe-0/0/1 priority-cost 25;

set track interface xe-0/0/2 priority-cost 10;

VRRP Configuration (cont)

• VRRP failover in 2 seconds or less – though configurable with advertise-interval setting

• VRRP tracks local interfaces--can’t be easily tied to BGP state

• Subtract priority-cost from priority when link goes down – Router2 xe-0/0/1 is the preferred outbound interface

at priority 120

– If Router2 xe-0/0/1 goes down, Router1 xe-0/0/1 takes over

• 120 – 25 = 95 < Router1 xe-0/0/1 priority 100

BGP Configuration

Router 1 [edit protocols bgp]

set local-as 12345;

set group interwebs type external;

set group interwebs export bgpnets;

set group interwebs neighbor 172.16.1.2 peer-as 1000;

set group joesgarage type external;

set group joesgarage import lowerpref;

set group joesgarage export [ prepend5X bgpnets ];

set group joesgarage neighbor 10.1.1.2 peer-as 2000;

[edit policy-options]

set policy-statement bgpnets term to-allow from protocol [ direct static ];

set policy-statement bgpnets term to-allow from route-filter 192.168.1.0/24 exact;

set policy-statement bgpnets term to-allow then accept;

set policy-statement lowerpref then local-preference 90;

set policy-statement prepend5X then as-path-prepend “12345 12345 12345 12345 12345”;

BGP Configuration (cont)

• Router with fewest prepends wins for inbound traffic

• Use local preference (default 100) to steer traffic out preferred outbound provider

• BGP failover after (typical) 30 second reconvergence

• Prepending doesn’t protect from flapping – Web users just hit reload. VoIP customers hit speed

dial to customer support

• Throw in BGP community usage to fine tune specific carriers through specific interfaces

Useful Commands

• Show vrrp Interface State Group VR state VR Mode Timer Type Address

xe-0/0/0.0 up 0 master Active D 3.453 lcl 192.168.1.2

• Show bgp summary

• Show route protocol bgp <network>

• Show bgp group

• Show nearest open bar

2012

Questions?