jisc update network operations - networkshop44
TRANSCRIPT
01/05/2023
Network operations
Jisc update: Network operations
John SeymourDirector of Network operations
01/05/2023
Networkshop44 - Jisc update: Network Operations
Janet6 infrastructure update
Glasgow
Leeds
Bristol
Manchester
London Telecity Powergate
Birmingham Nottingham
London TH NorthLondon Telecity HXGlobal
Transit & Peering
» New fibre arc to Stirling and new PoP at Stirling
» New PoP at Norwich on the fibre arc to Hinxton
» New PoP at the Olympic Park, London
» Optical node at Bangor:PSBA northern link
» Fibre arc via Newcastleinto Durham and Teesside
» London: arcs for HEIsand Francis Crick
n x 10G
100G
Global Transit & Peering
Stirling
Norwich
Olympic Park
London TH West
Bangor
Durham&
Teesside
London & Francis Crick
01/05/2023
Network operations
Some numbers
»Fibre added to the Janet6 infrastructure = 1287km»New Janet regional PoPs commissioned = 12»Regional core routers newly installed or replaced = 45»New or upgraded customer connections ~ 400»Managed Router Service: 58 new sites; total = 624
Janet core and regional expansion since March 2015
01/05/2023
Networkshop44 - Jisc update: Network Operations
Regional insourcing programmeRegion Date
movedAbMAN 22 Jun 2010LMN 1 Nov 2010SWERN 11 Apr
2012FaTMAN 24 Apr
2012LeNSE 1 Oct 2012NNW 1 Aug 2013CANLMAN 1 Oct 2013NorMAN 14 Nov
2013NIRAN 22 Nov
2013EMMAN 16 Dec
2013YHMAN 1 Sep 2014
» ClydeNET and EaStMAN insourced Oct last year
» The 5-year programme is now complete
» Much work still to do to make Janet a single homogeneous network
» New networks are at various stages of rollout in East, NW and Y&H
» London and SW Scotlandare now complete
ClydeNET 1 Oct 2015 EaStMAN 1 Oct 2015
01/05/2023
Network operations
Janet external connectivityNetflix
VoicenetAkamai
Virgin Radio
Bogons
Logicalis UKPipex / GXN
BBC
Datahop
InTechnology
INUK
Simplecall
LINX multicast
Gamma
Simplecall
Redstone
Updata
AQL 3G
Voicenet
Limelight
Limelight
AkamaiBTnet
Init7
Amazon
Microsoft EU (via TN)
Telekom Malaysia
Globelynx
10Gbit/s 1Gbit/s
100Gbit/s
GÉANT
GÉANT+
Microsoft EU (via TW)
Akamai
VM for LGfL
InTechnology
NHS N3
Exa Networks
Synetrix
BBC (HD 4Kpilots)
One Connect
Glasgow&
Edinburgh
HEAnet
BBC (Pacific Quay)
Gamma
BBC (HD 4K pilots)
NHS N3
SWAN(Glasgow)
SWAN(Edinburgh)
Telecity
Harbour
Exch.
Telehouse
North & West
VM for LGfLRM for
Schools
VM for LGfL
RM for Schools
Global Transit
Tata
IXLeeds
Global Transit Level3
Global Transit Level3
Microsoft EU
AQL 3G
Leeds
Manchester
China Telecom
China Unicom
Global Transit
TeliaSonera
IXManchester
LINX
Public Exchanges
~ 500 peerings~ 300 organisations
01/05/2023
Networkshop44 - Jisc update: Network Operations
TeleglobeAbilene
ESNET60
Hudson St
NYC
Janet
Global transit strategy (1)»Late 1990s: had our own
circuits to New York and a PoP at 60 Hudson St
»But expensive and complex:› 6 x 155M circuits
»By 2002, it was economic to purchase global transit from a “Tier 1” ISP
»Problem: resilience
LondonTelehouse
North
Global TransitSupplier A
01/05/2023
Networkshop44 - Jisc update: Network Operations
Janet
Global transit strategy (2)
TelecityLondon
HX
LondonTelehouse
North
Global TransitSupplier A
»Solution: a second connection to the Service Provider with independent access to the internet
»Still dependent on a single supplier
»Problem: the supplier went bust
01/05/2023
Networkshop44 - Jisc update: Network Operations
Global TransitSupplier B
Janet
Global transit strategy (3)
TelecityLondon
HX
LondonTelehouse
North
Global TransitSupplier A
» Solution: a second Tier 1 ISP with independent primary and second access to the internet
»But it’s very “London-centric”
»Problem: at the time, so were the ISPs
01/05/2023
Networkshop44 - Jisc update: Network Operations
Janet
Global transit strategy (4)Global Transit
Supplier C
TelecityManchest
erKH
LondonTelehouse
North
Global TransitSupplier A
Global TransitSupplier B
TelecityLondon
HX
» Solution: a Tier 1 ISP with independent access to the internet in the North
»Advantage: off loads traffic on Janet North-South legs
»Problem: so far so good:› Upgrades: multiple 10G› Procurement: staggered
2-year contracts
01/05/2023
Network operationsJanet Network DDoS experience
Global transit usage / cost
£ 0
£ 20 ,0 0 0
£ 40 ,0 0 0
£ 60 ,0 0 0
£ 80 ,0 0 0
£ 10 0 ,0 00
£ 12 0 ,0 00
£ 14 0 ,0 00
£ 16 0 ,0 00
£ 18 0 ,0 00
£ 20 0 ,0 00
0
2 ,0 00 ,0 0 0, 00 0
4 ,0 00 ,0 0 0, 00 0
6 ,0 00 ,0 0 0, 00 0
8 ,0 00 ,0 0 0, 00 0
1 0, 00 0 ,0 00 ,0 0 0
1 2, 00 0 ,0 00 ,0 0 0
1 4, 00 0 ,0 00 ,0 0 0
UsageCost
01/05/2023
Network operations
Janet external connectivityTotal external connectivity >
1 Tbit/sNetflix
VoicenetAkamai
Virgin Radio
Bogons
Logicalis UKPipex / GXN
BBC
Datahop
InTechnology
INUK
Simplecall
LINX multicast
Gamma
Simplecall
Redstone
Updata
AQL 3G
Voicenet
Limelight
Limelight
AkamaiBTnet
Init7
Amazon
Microsoft EU (via TN)
Telekom Malaysia
Globelynx
10Gbit/s 1Gbit/s
100Gbit/s
GÉANT
GÉANT+
Microsoft EU (via TW)
Akamai
VM for LGfL
InTechnology
NHS N3
Exa Networks
Synetrix
BBC (HD 4Kpilots)
One Connect
Glasgow&
Edinburgh
HEAnet
BBC (Pacific Quay)
Gamma
BBC (HD 4K pilots)
NHS N3
SWAN(Glasgow)
SWAN(Edinburgh)
Telecity
Harbour
Exch.
Telehouse
North & West
VM for LGfLRM for
Schools
VM for LGfL
RM for Schools
Global Transit
Tata →Telia
IXLeeds
Global Transit Level3
Global Transit Level3
Microsoft EU
AQL 3G
Leeds
Manchester
China TelecomChina Unicom
Public Exchanges
~ 500 peerings~ 300 organisations
IXManchester
LINX
01/05/2023
Network operations
Janet resilience and security
» Denial of Service attacks are on the increase› Janet DDoS incidents from CSIRT
records» 2 attacks a day somewhere on Janet» Engineers involved in dealing with 2
a week» Some of these we’ve seen as high as
50Gbit/s› At that level it’s not just a site
that’s hit but a whole region» What happened to Janet in
December and how we are working to protect Janet are the subject of later sessions.
126
465 527
2013 2014 2015
01/05/2023
Network operations
jisc.ac.uk
Network operations
Jisc Technologies
John SeymourDirector of Network [email protected]