Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

Robust Inter-Domain Routing

Addressing Systemic Vulnerabilities in BGP

Doug Montgomery (dougm@nist.gov) Manager, Internet and Scalable Systems Research

https://www.nist.gov/itl/antd/internet-scalable-systems-research

Systemic Vulnerabilities

• Faults / Accidents • Attacks

2

WIRED: HOW A TINY ERROR SHUT OFF THE INTERNET FOR PARTS OF THE US

https://www.wired.com/story/how-a-tiny-error-shut-off-the-internet-for-parts-of-the-us/

Targeted Internet Traffic Misdirection

https://dyn.com/blog/mitm-internet-hijacking/

Broad Range of Threats and Motivations • Financially motivated attacks

• Other motivations …

3

https://www.secureworks.com/research/bgp-hijacking-for-cryptocurrency-profit

https://arstechnica.com/information-technology/2017/04/russian-controlled-telecom-hijacks-financial-services-internet-traffic/

https://arstechnica.com/information-technology/2010/11/how-china-swallowed-15-of-net-traffic-for-18-minutes/

https://dyn.com/blog/iran-leaks-censorship-via-bgp-hijacks/

Broad Range of Risks •  Attacks to leverage other vulnerabilities

•  Attacks to Disable Infrastructure

4

https://blog.thousandeyes.com/amazon-route-53-dns-and-bgp-hijack/ https://www.csoonline.com/article/3258748/security/the-mirai-botnet-explained-how-teen-scammers-and-cctv-cameras-almost-brought-down-the-internet.html

BGP Systemic Vulnerabilities

5

•  Threats •  Route hijacks

–  Steers traffic away from legitimate servers •  Address squatting

–  Hijacks a not-in-service prefix and sets up spam servers •  Route detours

–  Modifies path causing data to flow via the attacker •  Route leaks

–  Announces routes in violation of ISP policy.

– Ramifications –  Exploitations commonly result in outages, spam,

misrouting of data traffic, eavesdropping on user data, DDoS, etc.

Route Hijacks • Pakistan Telecom Hijack of YouTube - 2008

6

Hijack of Routes to Critical Infrastructure

7

April, 2018

https://blog.thousandeyes.com/amazon-route-53-dns-and-bgp-hijack/

Route Leaks – Violations of Implicit Policy • Hathway / Airtel Route Leaks of Google Prefixes

8

March2015-Incidentanalysis:http://research.dyn.com/2015/03/routing-leak-briefly-takes-google/

Scale of Problem Space • Global Routing System • Recurring Threats

•  Solution Constraints •  Behavioral Impacts •  Installed base •  Economic Incentives •  Business / trust models

9

https://securityintelligence.com/bgp-internet-routing-what-are-the-threats/

https://blog.apnic.net/2018/01/10/bgp-in-2017/

State of the Solutions Space • BGP Origin Validation (BGP-OV)

•  Global public key infrastructure and protocol elements to enable BGP routers to verify that the origin AS in a BGP update, was authorized by the prefix owner to announce that route.

• BGP Path Validation (BGP-PV) •  Leverages the same PKI to enable each AS to digitally sign a BGP update,

proving that each AS in the PATH authorized the route announcement to its next hop.

• BGP Route Leak Detection / Mitigation •  BGP protocol modifications to allow networks to detect that a BGP routed path

violates typical customer-provider-peer policies for route redistribution.

10

BGP Origin Validation (BGP-OV) • BGP Origin Validation (BGP-OV)

•  Core specifications are complete •  IETF SIDR Working Group - https://datatracker.ietf.org/wg/sidr/documents/ •  IETF SIDR Ops WG - https://datatracker.ietf.org/wg/sidrops/documents/

•  Commercial implementations and production services exists. •  All 5 RIRs operate production RPKI services •  Commercial and opensource routers support RPKI-based origin validation.

•  Initial pilot and operational deployments are emerging. •  New specifications and implementations are emerging for:

•  Deployment optimizations and implementation clarifications •  Extensions to support additional security services.

11

BGP Origin Validation Components

12

Certification •  Securing routes to

your addresses •  Get certificates for your

address space •  Sign and publish ROAs

Two Sides of RPKI

13

Resource Public Key Infrastructure

Origin Validation •  (Securing your

routes to others’addresses) •  Retrieve ROAs from

other CA repositories •  Validate received routes

against the RPKI data

Two Sides of RPKI

14

•  RPKI Certificate Hierarchy

•  Rooted trust anchors at each RIR

•  Sub allocations represented by CA certificates.

•  ROAs signed by certificate holders.

•  RPKI Objects published in repository.

•  Hosted Model •  All RPKI operations hosted by

RIR.

RPKI Resource Certificates

15

•  Delegated Model •  Up / Down protocol to

register resources. •  Users operate their own

RPKI Certificate Authorities. •  Publication protocol to

publish RPKI objects •  Operates own RPKI

repository or uses public aggregator.

RPKI Resource Certificates

16

Validation in single AS •  Local RPKI validating

caches synchronize with global repositories.

•  Caches do all crypto / PKI validation operations.

•  Routers only receive a digested lists of ROA data.

•  No crypto on the router!

RPKI Origin Validation

17

Regional RPKI Services • Production RPKI services in RIRs:

•  AFRINIC: •  http://afrinic.net/en/initiatives/rpki-certification

•  APNIC: •  http://www.apnic.net/services/services-apnic-provides/resource-certification

•  ARIN •  https://www.arin.net/resources/rpki/

•  LACNIC: •  https://rpki.lacnic.net/rpki/

•  RIPE NCC: •  http://www.ripe.net/certification/

18

•  To use RPKI data for BGP origin validation, you will want to deploy one or more “validating caches”. •  These tools collect and cache

global RPKI data, perform X.509 validation on the objects,

•  … and then provides a highly summarized version to eBGP speaking routers.

•  The RPKI-to-RTR protocol enables eBGP routers to download this processed data for route filtering.

•  Multiple open source validating cache implementations are available!

RPKI Validating Caches

19

•  RPKI Origin Validation requires a router that can: •  Interface with a RPKI validating cache to

download lists of authorized origins: •  <prefix, max_length, origin_AS>, …..

•  Match incoming BGP updates against the list of authorized origins.

•  Enforce local policies based upon on the results of these matches: •  Valid, Invalid, Unknown

•  Major router vendors support these capabilities in shipping products today!

RPKI Router Implementations

20

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

Implementations & Tools •  RPKI Infrastructure •  RIPE validator 2

•  https://www.ripe.net/manage-ips-and-asns/resource-management/certification/tools-and-resources

•  RIPE validator 3 •  https://github.com/RIPE-NCC/rpki-validator-3/wiki

•  Routinator - Open source Relying Party - NLnet Labs •  https://github.com/NLnetLabs/routinator

•  RPKI.net Open Source Implementation of RPKI Tools •  https://github.com/dragonresearch/rpki.net/

•  RPSTIR - BBN Validation Tool •  https://sourceforge.net/projects/rpstir/

•  Router Implementations •  Cisco

•  https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/command/irg-cr-book/bgp-m1.html#wp3677719851

•  Juniper •  https://www.juniper.net/documentation/en_US/

junos12.2/topics/topic-map/bgp-origin-as-validation.html •  Nokia

•  https://infoproducts.alcatel-lucent.com/cgi-bin/dbaccessfilename.cgi/9300731102_V1_7750%20SR%20OS%20Router%20Configuration%20Guide%2012.0.R4.pdf

•  Quagga / FRR, BIRD •  https://www.nist.gov/services-resources/software/bgp-

secure-routing-extension-bgp-srx-prototype •  http://rtrlib.realmv6.org/

•  Go BGP •  https://github.com/osrg/gobgp

21

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

RPKI Based Origin Validation Trends • Significant Deployments

•  RIPE •  https://rpki-monitor.antd.nist.gov/?

p=2&s=0 •  Implementation of RPKI and IRR

filtering on the AMS-IX platform •  https://www.ripe.net/support/training/

ripe-ncc-educa/presentations/use-cases-stavros-konstantaras.pdf

•  Cloudflare

•  https://blog.cloudflare.com/rpki-details/

•  Ongoing BGP-OV Specifications •  SIDROPs WG

•  https://datatracker.ietf.org/wg/sidrops/documents

•  Autonomous System Provider Authorization

•  https://datatracker.ietf.org/doc/draft-azimov-sidrops-aspa-profile/

•  https://datatracker.ietf.org/doc/draft-azimov-sidrops-aspa-verification/

•  Drop Invalid if Still Routable •  https://datatracker.ietf.org/doc/draft-

sriram-sidrops-drop-invalid-policy/ •  Cloudflare GoRTR

•  https://github.com/cloudflare/gortr

22

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

BGP Path Validation (BGP-PV) • BGPSec (BGP-PV)

•  Core specifications are complete •  IETF SIDR Working Group - https://datatracker.ietf.org/wg/sidr/documents/ •  IETF SIDR Ops WG - https://datatracker.ietf.org/wg/sidrops/documents/

•  RFC 8205 – BGPsec Protocol Specification •  RFC 8208 – BGPsec Cryptographic Algorithms •  RFC 8206 – Operational BCP •  RFC 8374 – BGPsec design rationale / discussion

•  Commercial implementations and production services are lagging. •  Two open source reference implementations exist.

•  Ongoing Research in Performance Optimizations and Deployment Architectures.

23

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

BGPSec Design • BGP Path Validation (BGP-PV)

•  RPKI Support for Router Keys. •  Validating Cache’s provide public keys to routers. •  BGP-PV routers digitally sign and validate each hop in the BGP path.

•   “BGPsec Protocol Specification”, IETF RFC 8205.•  "BGPsec Algorithms, Key Formats, and Signature Formats", IETF RFC 8208.

•  Crypto on the router!

24

BGP-Secure Routing eXtension (BGP-SRx) •  Open Source Reference Implementation

•  Software router with extensions for: RPKI/RTR protocol, maintenance ROA distilled data, ROV and RPKI-aware BGP route policies.

•  Full support of BGPSec path validation •  Designed to support experimentation with different

architectural configurations of SRx and RPKI components and different trade-offs performance and router impact.

•  BGP-SRx Status •  SRx Server •  SRx API •  Quagga SRx (integrates SRx API into Quagga

router) •  src & yum repository:

•  https://www-x.antd.nist.gov/bgpsrx/

25

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

BGPsec Performance Optimization • Protocol Optimizations

•  "Design and analysis of optimization algorithms to minimize cryptographic processing in BGP security protocols.", Computer Communications.

• Cryptographic

Optimizations •  "

High Performance BGP Security: Algorithms and Architectures(link is external)",NANOG69.

26

0

2,000

4,000

6,000

8,000

10,000

12,000

0 20 40 60 80 100 120

Num

berofVerificatio

ns

durin

gPe

akSecon

d

Time(minutes)

UnoptimizedCCS-ECBPO-EC

WorkloadinthePeak-SecondofeachMinuteduringa Two-HourTimePeriod

40,000

24,649

12,3258216

61624930

4108 3521 3081 2739 2465

1,000

10,000

100,000

1 2 3 4 5 6 7 8 9 10

Upd

ates

Pro

cess

ed p

er S

econ

d

AS Path Length (#Sigs in BGPsec Update)

Sign onlyVerify only

Xeon®CPUE3-1285v43.5GHz

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

SignP256 VerifyP256 SignP384 VerifyP384

Ope

ratio

nsperse

cond

OpenSSL1.1.0

taraEcCRYPT-3

Est

ablis

hing

the

Tech

nica

l Bas

is fo

r Tru

stw

orth

y N

etw

orki

ng

BGP Route Leak Detection

27

•  IETF IDR and Grow WG •  https://tools.ietf.org/html/draft-ietf-idr-route-leak-detection-mitigation

GeneralPrinciplesofDesignC:SolutionUsingBGPCommunities•  Considering Community based encoding of RLP info for faster adoption •  Wish to limit the number of RLP entries so that they can be accommodated in 1

or 2 Community attributes per update. Reason: Avoid having a long string of Community attributes per BGP update because the more they are, the lesser the chance that they will all make it through. If some get dropped, then the rest become useless. Also, save memory, simplify processing, and improve robustness.

•  Based on the analysis and knowledge we have so far about RLP/eOTC, independent of encoding (Attribute or Community), at the minimum the RLP info must include: Ø ASN of the RLP aware AS that most recently asserted that it sent update to a

customer or peer; let us call this DO = Down Only indication Ø Leak warning: L = Leak indication

Ø L = ASN of the first RLP aware AS in the path that is forwarding route from customer or lateral peer in spite of detecting a leak §  AS in question is avoiding unreachability (absence of alternative route)

Note: RLP = Route Leak Protection; DO and L together constitute RLP

IllustrationofDownOnly(DO)andLeak(L)indications–1of2

Note: RLP = Route Leak Protection; DO alone or DO and L together constitute RLP

1

2

[DO = 1]

6

3

P2Corp2pC2P

q

AS does not participate in RLP and starts/restarts a leak

n

Legend:

[DO = 1]

4

[DO = 1, L = 3]

[DO = 3, L = 3]

C2P

5[DO = 1, L = 3]

C2P

7

[DO = 6, L = 3]

P2C

P2C

Once a route is tainted with L = X, it remains tainted with the same (L = X) when it propagates. This is “stickiness” of L.

8[DO = 3, L = 3] p2p

9

[DO = 8, L = 3]

P2C

No alternative route for q

1[DO = 1]

2

q

AS does not participate in RLP and starts/restarts a leak

n

Legend:

3[DO = 1] p2p

4[DO = 3, L = 3]

P2C

P2CNo alternative route for q

5C2P

P2C

[DO = 5]

IllustrationofDownOnly(DO)andLeak(L)indications–2of2

1

2

q

3[DO = 2] p2p

4

[DO = 4, L = 4]

P2C

No alternative route for q

6

C2P

P2C

[DO = 6]

C2P[DO = 2]

p2p

5

Note: RLP = Route Leak Protection; DO alone or DO and L together constitute RLP

EncodingRLPinBGPCommunities

Relevant RFCs: RFC 4360: BGP Extended Communities Attribute RFC 7153: IANA Registries for BGP Extended Communities RFC 8092: BGP Large Communities Attribute