bas kreukniet, sr network specialist at surf sara nl-t1 expectations, findings, and innovation...
TRANSCRIPT
Bas Kreukniet, Sr Network Specialist at SURFSARA
NL-T1 Expectations, findings, and innovation
Geneva Workshop 10 Februari 2014
Outline
1. Expectations from NL-T1 grid administrators2. Findings while connecting to the LHCONE3. Innovation: Ethernet OAM and NSI
Expectations from NL-T1 administrators
Advice from grid administrators NL-T1:
• Bulk data with simple applications (grid-FTP). “Keep it simple” • Network provisioning from application is
considered “complex”• Don’t rush to merge LHCOPN with LHCONE
NL-T1 connected to LHCONE since 20 Jan 2014
BGP routing starts at the connected site.
BGP routing for T1’s:
Not only a “NREN thing” – it already starts at your organisation!
Focus on some BGP topics for connecting sites to LHC networks
LHCOPN connectivity
LHCONE connectivity
128.142.0.0/16 *[BGP/170] 1w2d 04:07:47, MED 120, localpref 100 AS path: 20965 20641 513 I > to 62.40.126.161 via xe-4/0/1.2012
NL-T1AS1162
GÉANTAS20965
LHCONE-RS CERN
AS20641
CERNAS513
LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both
Preferred route: LHCOPN, LHCONE or Internet
• primary: direct T1–T1 over LHCOPN• secondary: T1–T1 over LHCOPN via another T1 • tertiary: LHCONE• quaternary: Internet
Preferred route: route (a)symmetry
If everyone makes this choice, all connections are symmetric.But: sites may have different preferences:
10 Gb/s 10 Gb/s
1 Gb/s 100 Gb/ssite 1 site 2
ISP-B
ISP-A
Tie-breaker between LHCOPN and LHCONE
CNAF routers at NL-T1
LHCOPN131.154.128.0/17 *[BGP/170] 14:58:18, localpref 100 AS path: 34878 137 I
LHCONE131.154.128.0/17 *[BGP/170] 1w2d 04:23:40, MED 120, localpref 100 AS path: 20965 137 I
LHCOPN / LHCONE route preference
BNL routes at NL-T1
LHCOPN130.199.185.0/24 *[BGP/170] 2w5d 09:35:43, MED 10, localpref 100 AS path: 513 43 I > to 192.16.166.73 via xe-1/1/0.0 [BGP/170] 3d 01:15:22, MED 51, localpref 100 AS path: 39590 513 43 I > to 109.105.124.17 via xe-2/1/0.0
LHCONE130.199.185.0/24 *[BGP/170] 1w2d 04:27:43, MED 120, localpref 100 AS path: 20965 293 43 I > to 62.40.126.161 via xe-4/0/1.2012
Problems and concerns regarding BGP
• A site advertising his routes has no control who to send a route to. At best they can give hints with BGP communities.
• The site receiving a route decides which route to accept and how to accept.
BGP in LHC networks
Only As is not enough. More info needed. community for origin of a route T1/T2?Or even better: -site connected to LHCOPN-site connected to LHCONE- Site connected to both:Type A “prefer LHCONE for this route”Type B “prefer LHCOPN for this route”- Specials: dedicated link between two (T1) sites. (“VPN”or “private link”)
Route Preference Solutions
• Idea: Tag routes with two types of communities: • One for origin or source• One for destinations• See also: BGP hinting by Martin Sweeny (Indiana U)
• BGP Always-compare-MED always on. We sometimes add metrics on incoming routes.
Operational Issues
• We received routes over LHC from a site,but traffic we send is blackholed by that site
• The site was still reachable over the Internet.• This happened to us twice recently: on LHCOPN (accidental route
redistribution) and LHCONE (incoming IP filter).• Configuration errors will be made (we’re also just human)
• LHCOPN Link NL-T1 – TRIUMF link is still unstable• 31 outages last 4 months
Monitoring remains important!
LHCONE and LHCOPN layer 2 monitoring
Ethernet OAM monitoring or Layer 2 monitoring
NIKHEF, TRIUMF to participate as Measurement PointsLooking for T1’s and T2 to participate
Advantages:• Layer2 keep-alive, ping and traceroute• Interdomain, intervendor solution• L2 devices can be made visible• unidirectional fibercuts can be signalled
LHCONE Innovation
NL-T1 likes to partcipate in NSI for LHCONE
NSI experiences so far:• Use-case: Life Science Grid (LSG) in Holland
makes use of NSI since autumn 2013. • Freek contributed to standard• Sander wrote NSI client and implemented it for
“Cloud Bypassing” in the Life Science Grid.
Cloud Bypassing on Life Science Grid (LSG)
Compute clusters at ±10 locations in the Netherlands
Cloud Bypassing on Life Science Grid (LSG)
Offloading campus networks (some only have 1 Gb/s Internet)
LHCONE Innovation: Offloading is cheaper
Internet full routing (incl backup): € 8k – €10k per month for 10 Gb/sLHCONE or dynamic lightpath: € 2k – 3k per month for 10 Gb/s
Questions?
Erik Ruiter
Bas Kreukniet
Diederik Vandevenne
Sander Boele
Farhad Davani
Freek Dijkstra