draft - arcfire projectict-arcfire.eu/wp-content/uploads/2015/12/arcfire_d4.3.pdf3 experiment 1:...
TRANSCRIPT
DRAFTGrant Agreement No.: 687871
ARCFIRE
Large-scale RINA Experimentation on FIRE+
Instrument: Research and Innovation ActionThematic Priority: H2020-ICT-2015
D4.3 Design of experimental scenarios, selection of metrics and KPIs
Due date of Deliverable: Month 12Actual submission date: February 17, 2017
Start date of the project: January 1st, 2016. Duration: 24 monthsversion: V1.0
Project funded by the European Commission in the H2020 Programme (2014-2020)
Dissemination level
PU Public X
PP Restricted to other programme participants (including the Commission Services)
RE Restricted to a group specified by the consortium (including the Commission Services)
CO Confidential, only for members of the consortium (including the Commission Services)
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
H2020 Grant Agreement No. 687871
Project Name Large-scale RINA Experimentation on FIRE+
Document Name Deliverable 4.3
Document Title Design of experimental scenarios, selection of metrics andKPIs
Workpackage WP4
Authors Dimitri Staessens (imec)
Sander Vrijders (imec)
Eduard Grasa (i2CAT)
Leonardo Bergesio (i2CAT)
Miquel Tarzan (i2CAT)
Bernat Gaston (i2CAT)
Sven van der Meer (LMI)
John Keeney (LMI)
Liam Fallon (LMI)
Vincenzo Maffione (NXW)
Gino Carrozzo (NXW)
Diego Lopez (TID)
John Day (BU)
Editor Dimitri Staessens
Delivery Date December 31st 2016
Version v1.0
2
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
1 Abstract
This deliverable details the preparatory work done in ARCFIRE WP4 before the actual experimen-tation can begin. It takes the converged network operator scenarios for a traditional network designand the RINA designs that were developed in WP2 together with the testbed overview that wascompiled by T4.1 and extracts interesting reference scenarios for the 4 main experiment objectives.This document will thus serve as a crutch for the experimenters during experimentation, providingall necessary information on the experiment objectives and tools in one location, and providingreferences to the sections of the WP2 deliverables where more details can be found.
First, this document briefly summarises the WP2 reference scenarios for the converged networkoperator, focusing on the architecture of the access network, since that is where the technologydiversification is most apparent. The first experiment will investigate the differences between aRINA and an evolutionary 5G network in terms of management complexity, targeting configurationmanagement deploying a new service. The experiment has been divided into 4 sub-experimentswith green and brown field starting points for the service. The second experiment will performnetwork performance oriented measurements, pitting a 5G scenario based on LTE vs a 5G scenariobased on RINA. Here, ARCFIRE will evaluate network parameters such as overhead with respectto both data transport and mobility management, and scalability with respect to routing, also in thepresence of network failures. The third experiment turns our sight towards multi-provider networks.RINA will be evaluated towards its capability for maintaining end-to-end QoS guarantees withrespect to delay, jitter and bandwidth. Furthermore, it will evaluate how renumbering end-usersaddresses with respect to the location in the network improves overall scalability. Experiment 3will also delve into the OMEC scenario for RINA, keeping applications reachable while they movethrough the network. The fourth and final experiment brings ARCFIRE in the world of DDoSattacks. It will investigate the various checks in RINA’s flow allocation can bar malicious attackersfrom taking down critical services.
3
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Table of Contents
1 Abstract 3
2 Reference scenario 102.1 Baseline reference for traditional networks . . . . . . . . . . . . . . . . . . . . . 102.2 RINA network design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Experiment 1: Management of multi-layer converged service provider networks 163.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.1 The Current Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.1.2 Multi-layer Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . 173.1.3 The resulting Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 183.1.4 Scope: Configuration Management . . . . . . . . . . . . . . . . . . . . 183.1.5 Aspects, KPIs, Scenarios and Use Cases . . . . . . . . . . . . . . . . . . . 21
3.2 Experiment 1-1: Deploy Network from Zero . . . . . . . . . . . . . . . . . . . . 263.2.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.2.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.2.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.5 Experiment scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Experiment 1-2: Deploy new Service in Existing Network . . . . . . . . . . . . 303.3.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.3.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3.5 Experiment scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Experiment 1-3: Deploy Network and Service from Zero . . . . . . . . . . . . . 333.4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.4.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.4.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.4.5 Experiment scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.4.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.5 Experiment 1-4: Add new Network Node - Zero touch . . . . . . . . . . . . . . 383.5.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.5.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.5.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
3.5.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.5.5 Experiment scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.5.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4 Experiment 2: Deploying resilient, virtualised services over heterogeneous physicalmedia 434.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.3 Experiment scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.4 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.5 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.6 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.7 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5 Experiment 3: End to end service provisioning across multiple network providers 515.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2 Quality of Service levels guaranteed by service provider network . . . . . . . . . . 51
5.2.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.2.5 Experiment scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.2.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.3 Dynamic and seamless DIF renumbering . . . . . . . . . . . . . . . . . . . . . . 575.3.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.3.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.3.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595.3.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.3.5 Experiment scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.3.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.4 Application discovery, mobility and layer security in support of OMEC . . . . . 655.4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.4.2 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665.4.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665.4.4 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.4.5 Experiment scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.4.6 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
6 Experiment 4: Studying the effects of (Distributed) Denial of Service (D)DoS attacksinside and over RINA networks 736.1 Introduction and motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.1.1 State of the art in botnet formation . . . . . . . . . . . . . . . . . . . . . 736.1.2 State of the art of DDoS attacks . . . . . . . . . . . . . . . . . . . . . . 746.1.3 State of the art of mitigation techniques of DDoS attacks . . . . . . . . . 76
6.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786.3 Experiment scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.4 Metrics and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.5 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.6 Testbeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.7 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
7 Conclusion 89
6
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
List of Figures
1 Converged service providers’ network . . . . . . . . . . . . . . . . . . . . . . . 102 Residential fixed Internet access: data plane (up) and control plane (down) . . . . . 113 4G Internet access: data plane (up) and control plane (down) . . . . . . . . . . . 124 Layer 3 VPN between customer sites: data plane (up) and control plane (down) . 135 Fixed access network (RINA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Cellular access network (RINA) . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Enterprise VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Management Points in a Protocol Stack Pattern . . . . . . . . . . . . . . . . . . 169 Management Points in a Layered Domain Pattern . . . . . . . . . . . . . . . . . 1910 FCAPS, Strategies, and RINA Network . . . . . . . . . . . . . . . . . . . . . . 1911 DMS: Stand-alone System for Benchmark Experiments . . . . . . . . . . . . . . 2412 DMS: Full System for Experiments with a RINA Network . . . . . . . . . . . . 2513 Experiment 1: Minimal RINA System for simple Experiments . . . . . . . . . . 2514 Baseline LTE experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4515 End-to-end LTE experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4616 Guaranteed QoS levels experiment: abstract layering structure, showing the differ-
ent DIFs to be considered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5217 Guaranteed QoS levels experiment: abstract layering structure, showing the differ-
ent DIFs to be considered. Provider edge routers are shown . . . . . . . . . . . . 5218 Guaranteed QoS levels experiment: small DIF experiment, backbone DIF connectivity 5419 Guaranteed QoS levels experiment: layering of the scenario . . . . . . . . . . . . 5620 Renumbering experiment: small single DIF scenario, layering structure (up), and
DIF connectivity (down) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6121 Renumbering experiment: large single DIF scenario, DIF connectivity . . . . . . 6222 Renumbering experiment: DIF structure for the multi-DIF scenario . . . . . . . . 6323 Renumbering experiment: systems for small scale multi-DIF scenario . . . . . . 6324 Renumbering experiment: systems for large scale multi-DIF scenario . . . . . . 6425 Categorisation of scenarios and configurations for the renumbering experiments . 6426 OMEC experiment: physical systems involved in the scenario . . . . . . . . . . 6827 OMEC experiment: DIF configurations: UE to server on the public Internet . . . 6928 OMEC experiment: DIF configurations: UE to server on the provider’s cloud . . 7029 Vulnerabilities that enable malware to take control of Internet-connected IoT devices 7430 Classification of DDoS defense mechanisms depending on their deployment . . . 7631 Experiment 4 scenario 1: single top-level DIF (public Internet) configuration . . 8032 Experiment 4 scenario 2: multiple top-level DIFs configuration . . . . . . . . . . 8033 Models for the different DIFs present in the DDoS experiments . . . . . . . . . . . 8134 Systems in the small scale experimental scenario . . . . . . . . . . . . . . . . . 8235 Systems in the large scale experimental scenario . . . . . . . . . . . . . . . . . . 83
7
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
36 Categorisation of DDoS experimental scenarios . . . . . . . . . . . . . . . . . . 84
8
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
List of Tables
1 KPIs for experiment 1-1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 Software required for experiment 1-1 . . . . . . . . . . . . . . . . . . . . . . . . 293 Milestones for experiment 1-1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 KPIs for experiment 1-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Software required for experiment 1-2 . . . . . . . . . . . . . . . . . . . . . . . . 326 Milestones for experiment 1-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 KPIs for experiment 1-3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 Software required for experiment 1-3 . . . . . . . . . . . . . . . . . . . . . . . . 379 Milestones for experiment 1-3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 3810 KPIs for experiment 1-4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3911 Software required for experiment 1-4 . . . . . . . . . . . . . . . . . . . . . . . . 4012 Milestones for experiment 1-4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4113 KPIs for experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4714 Software for experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4815 Testbeds for experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4816 Planning of experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4917 KPIs for guaranteed QoS levels experiment . . . . . . . . . . . . . . . . . . . . 5318 Software required for guaranteed QoS levels experiments . . . . . . . . . . . . . 5419 KPIs for guaranteed QoS levels experiments . . . . . . . . . . . . . . . . . . . . 5620 Milestones for guaranteed QoS levels experiment . . . . . . . . . . . . . . . . . 5721 KPIs for renumbering experiments . . . . . . . . . . . . . . . . . . . . . . . . . 5922 Software required for renumbering experiments . . . . . . . . . . . . . . . . . . 5923 Testbeds for renumbering experiments . . . . . . . . . . . . . . . . . . . . . . . 6024 Milestones for renumbering experiments . . . . . . . . . . . . . . . . . . . . . . 6525 KPIs for OMEC experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6626 Software required for OMEC experiments . . . . . . . . . . . . . . . . . . . . . 6727 Testbeds for OMEC experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 6728 Milestones for renumbering experiments . . . . . . . . . . . . . . . . . . . . . . 7229 KPIs for DDoS attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8530 Software required for Experiment 4 . . . . . . . . . . . . . . . . . . . . . . . . 8731 Testbeds used in experiment 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 8832 Milestones for DDoS experiments . . . . . . . . . . . . . . . . . . . . . . . . . 88
9
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
2 Reference scenario
All experiments in WP4 are grafted in the converged service provider network design as researchedin WP2. This section provides a quick summary of the core designs that will be used in theexperiments. More details can be found in deliverables D2.1 [1] and D2.2 [2].
2.1 Baseline reference for traditional networks
xDSL FTTH WiFi 4G
National DC Regional DC(s)
Metropolitan aggregation
Metropolitan aggregation
… …
Metro DC
Metro DC
Access
Internet Border
Private peering with other operators , or Internet transit
To Internet eXchange Point (IXP)
IP eXchange border To IPX network (IMS traffic)
micro DC
Metropolitan aggregation Metropolitan
aggregation
Metropolitan aggregation
Metropolitan aggregation
Metropolitan aggregation
Core/backbone
Figure 1: Converged service providers’ network
Figure 1 recaps the main parts of a converged service provider network. The network ispartitioned in three: several types of access networks allow the provider to reach its customersvia wired and wireless technologies. The traffic from these access networks is aggregated bymetropolitan area networks (MANs), which aggregate the traffic towards the network core.
Customers and services are identified and authenticated in the access or MAN. At the core, thetraffic is either forwarded to a DC or an Interconnect edge router (e.g. Internet edge) otherwise.The service provider may have different datacentres attached to different parts of the network:
• Micro data centres attached to the access networks, supporting the Mobile Edge Computingconcept by running very latency-critical services very close to its customers. Micro-data-centers may also be used to support C-RAN (Cloud RAN) deployments.
• Metro data centres attached to the metropolitan networks. These DCs could host serviceproviders’ network services such as DHCP, DNS or authentication servers; but also ContentDelivery Networks (CDNs) or even provide cloud computing services to customers.
• Regional or national data centres attached to the core networks. Could run the sameservices as metro data-centers at a national scale, as well as mobile network gateways and/orthe Network Operations Center (NOC).
10
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
DSLAM
XDSL
AAL5/ATM
CPE BRAS/Edge router Carrier Eth.
PE Carrier Eth.
PE
802.3
802.1ah
802.1q/p or 802.1ad
Eth PHY Eth PHY
Eth PHY Eth PHY
Carrier Eth. P
802.3
Eth PHY
PPP and PPPoE
Host L2/L1
IP (Internet)
Core router Core router
L2/L1 L2/L1
MPLS (LSP)
Internet Border Router
Internet Border Router
802.3
Eth PHY
Customer network Service Prov. 2 Service Prov. 1 network
Access
Aggregation Service Edge Core (BGP-free) Internet Edge
Internet eXchange Point Core PoP, city B Core PoP, city A City A Ethernet-based MAN City A Cabinets
MPLS (VPN/Pseudo Wire)
TCP/UDP
DSLAM
XDSL
AAL5/ATM
CPE BRAS/Edge router Carrier Eth.
PE Carrier Eth.
PE
802.3
IEEE 802.1aq (SPB)
802.1q/p or 802.1ad
Eth PHY Eth PHY
Eth PHY Eth PHY
Carrier Eth. P
802.3
Eth PHY
PPP and PPPoE
Host
L2/L1
Core router Core router
L2/L1 L2/L1
IP (operator)
Internet Border Router
Internet Border Router
802.3
Eth PHY
Customer network Service Prov. 2 Service Prov. 1 network
Access
Aggregation Service Edge Core (BGP-free) Internet Edge
Internet eXchange Point Core PoP, city B Core PoP, city A City A Ethernet-based MAN City A Cabinets
TCP
IS-IS IS-IS IS-IS
iBGP
RSVP (TE) IP
TCP
eBGP
Figure 2: Residential fixed Internet access: data plane (up) and control plane (down)
Figure 2 shows the data and control plane for fixed access. The design shows a Carrier-Ethernetbased MAN aggregating traffic and forwarding it to one or more BRAS routers located at a corePoP. In the control plane, a BRAS (connected over PPP) authenticates customers. Shortest PathBridging (SPB) in the MAN enables traffic engineering. eBGP is used by the provider border routerto exchange traffic with its peer or upstream routers. Routes are disseminated to the BRAS(es) viaiBGP. The BGP-free core runs an MPLS control plane.
11
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
DSLAM eNodeB
Multi Service Edge Carrier Eth.
PE Carrier Eth.
PE
802.1ah
802.1q/p or 802.1ad
Eth PHY Eth PHY
Eth PHY Eth PHY
Carrier Eth. P
MAC
PHY
UE
802.3
Eth PHY
P-GW
Service Prov. 1 network
Access Core (BGP-free) Internet Edge
Core PoP, city B Core PoP, city A
City A Ethernet-based MAN City A Antenna sites
RLC
PDCP
S-GW
IP (provider)
UDP
GTP-U
Eth PHY
802.3
UDP
GTP-U
IP (Internet)
Aggregation
L2/L1
Core router Core router
L2/L1 L2/L1
MPLS (LSP)
Internet Border Router
MPLS (VPN/Pseudo Wire)
802.3
Eth PHY
Service Edge Mobile gateways
DSLAM
eNodeB Multi Service Edge Carrier Eth.
PE Carrier Eth.
PE
802.1aq (SPB)
802.1q/p or 802.1ad
Eth PHY Eth PHY
Eth PHY Eth PHY
Carrier Eth. P
MAC
PHY
UE 802.3
Eth PHY
P-GW
Service Prov. 1 network
Access Core (BGP-free) Internet Edge
Core PoP, city B Core PoP, city A
City A Ethernet-based MAN City A Antenna sites
RLC
PDCP
S-GW
IP (provider)
Eth PHY
802.3
UDP
GTP-C
Aggregation Service Edge Mobile gateways
RRC
802.3
Eth PHY
MME
S1-AP
NAS
SCTP GTP-C
UDP
L2/L1
Core router Core router
L2/L1 L2/L1
IP (operator)
Internet Border Router
802.3
Eth PHY
TCP
IS-IS IS-IS IS-IS
iBGP
RSVP (TE) IP
TCP
eBGP
Figure 3: 4G Internet access: data plane (up) and control plane (down)
Figure 3 shows the data (user) plane and control plane for wireless (4G) access. The eNodeB(base station) is attached to the aggregation network, which aggregates the traffic from multiplebase stations into a Core PoP. The core PoP contains a Multi Service Edge and forwards the trafficto the mobile network gateways forming the EPC (Evolved Packet Core). In the example, the S-GWand the P-GW are located at the core PoP. Internet traffic reaching the P-GW is forwarded to one ofthe provider’s Internet border routers through the core network. In the control plane. the UE runsthe NAS protocol against the Mobility Management Element (MME), which allows the MME toauthenticate the user and negotiate handovers. The P-GW runs iBGP with Internet border routers,allowing it to route UE traffic to the Internet.
12
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
CPE MS Edge router
Carrier Eth. PE
Carrier Eth. PE
802.1ah
802.1q/p or 802.1ad
Eth PHY
Eth PHY Eth PHY Eth PHY
Carrier Eth. P
802.3
Eth PHY
Host L2/L1
IP (VPN)
Core router Core router
L2/L1 L2/L1
MPLS (LSP)
MS Edge Router
Customer network Service Prov. 1 network
Aggregation Service Edge Core (BGP-free)
Core PoP, city B Core PoP, city A City A Ethernet-based MAN
MPLS (VPN/Pseudo Wire)
TCP/UDP
Carrier Eth. PE
Carrier Eth. PE
802.1ah
Eth PHY Eth PHY
Carrier Eth. P
802.1q/p or 802.1ad
Eth PHY
802.3
Eth PHY Eth PHY
Aggregation Service Edge
Customer network
City B Ethernet-based MAN
Host
CPE
CPE
MS Edge router
Carrier Eth. PE
Carrier Eth. PE
802.1aq (SPB)
802.1q/p or 802.1ad
Eth PHY
Eth PHY Eth PHY Eth PHY
Carrier Eth. P
Core router Core router MS Edge Router
Customer network Service Prov. 1 network
Aggregation Service Edge Core (BGP-free)
Core PoP, city B Core PoP, city A City A Ethernet-based MAN
Carrier Eth. PE
Carrier Eth. PE
802.1aq (SPB)
Eth PHY Eth PHY
Carrier Eth. P
802.1q/p or 802.1ad
Eth PHY Eth PHY
Aggregation Service Edge
Customer network
City B Ethernet-based MAN
CPE L2/L1 L2/L1 L2/L1
IP (operator)
TCP
IS-IS IS-IS IS-IS
iBGP
RSVP (TE)
IP
TCP
eBGP
IP
TCP
eBGP
Figure 4: Layer 3 VPN between customer sites: data plane (up) and control plane (down)
Figure 4 illustrates data and control plane for a Layer 3 VPN service between two businesssites. The customer CPE router is directly attached to the MAN. The L3 VPN service is carriedover a VLAN to the MS Edge router, which is running a Virtual Routing and Forwarding (VRF)instance to forward the VPN traffic through the MPLS core. When VPN traffic exits the MPLScore, another VRF instance forwards it through another VLAN over the MAN delivering it to theCPE at the other business site. The control plane runs eBGP between the CPE and the MS Edgerouters to exchange VPN route information. These VPN routes are disseminated over the core viaiBGP, allowing all VPN locations to learn the routes required to forward the VPN traffic across allsites.
13
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
2.2 RINA network design
This section summarises the RINA network design for the 3 scenarios above.
Host
CPE
PtP DIF
Home DIF
Public Internet or VPN or app-specific DIF
Access Router
Edge Service Router
PtP DIF . . . PtP DIF
Host
Service provider top level DIF
PtP DIF Aggregation
DIFs
Customer network
Provider network
Local loop Aggregation
. . .
Figure 5: Fixed access network (RINA)
Figure 5 shows a RINA network structure for a fixed access network. The CPE is connected tothe access router via a point-to-point DIF. This DIF provides IPC services to one or more serviceDIFs, which are used to authenticate the customer and support access to one or more utility DIFs,such as a public Internet DIF or VPN or application-specific DIFs. The traffic from multiple accessrouters is multiplexed over the aggregation network into an edge service router, which forwards itfurther towards its final destination.
IP (e.g. Internet)
TCP or UDP
PDCP GTP-U
Protocol conversion
GTP-U
RLC
MAC
L1
UDP
IP (LTE transport)
MAC MAC . . .
L1 . . . L1
UDP
IP (LTE transport)
MAC MAC . . .
L1 . . . L1 UE
eNodeB S-GW P-GW
EPS bearer EPS bearer
LTE-Uu
S1-U S5/S8
MAC
L1
SGi
Public Internet DIF
Radio DIF . . .
. . . . . . UE
eNodeB S-GW P-GW
LTE-Uu
S1-U S5/S8 SGi
PtPDIF
Metro DIF
PtPDIF PtPDIF
PtPDIFMobile Network Top Level DIF
Backbone DIF
App
App
PtPDIF
Figure 6: Cellular access network (RINA)
Figure 6 shows a possible RINA network for mobile access. A radio multi-access DIF managesthe radio resource allocation and provides IPC over the wireless medium. A Mobile networktop-level DIF provides flows spanning the scope of the mobile network, where the Metro and
14
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Backbone DIFs multiplex and transport the traffic of the Mobile Network Top-Level DIF. Finally,the public Internet DIF allows applications in the UE to connect to other applications available onthe public Internet.
PE-rs 1
IPCP
PE-rs 2 P2 P1 CE 2 CE 1 H1 H2
Customer 1 Site 1 Network
Service Provider Network
Customer 1 Site 2 Network
MTU-s 1
IPCP IPCP
MTU-s 2
IPCP IPCP
P (metro) P (metro)
IPCP
App App
PtP DIF
Metro DIF Backbone DIF Metro DIF
Green VPN DIF
VPN Service DIF
PtP DIF PtP DIF PtP DIF PtP DIF PtP DIF
PtP DIF
PtP DIF
PtP DIF PtP DIF PtP DIF
PE-rs 1
VPLS
PE-rs 2 P2 P1 CE 2 CE 1 H1 H2
Customer 1 Site 1 Network
Service Provider Network
Customer 1 Site 2 Network
MTU-s 1
VPLS VPLS
MTU-s 2
VPLS VPLS
P (metro) P (metro)
VPLS
App App TCP
IP
802.3
Eth/Net Eth/Net
MPLS
MPLS
Eth/Net Eth/Net Eth/Net
MPLS
MPLS
Eth/Net Eth/Net
MPLS
MPLS
802.3 802.1q
PBB (802.1ah)
Protocol conversion
Figure 7: Enterprise VPNs
Figure 7 shows a RINA configuration for providing Enterprise VPN services. The variousrouters are connected over various Point-to-Point DIFs. The Metro DIF provides IPC over metroaggregation networks, multiplexing the traffic of the different types of services the operator providesover the metropolitan segment. A backbone DIF provides IPC over the core segment of the network,interconnecting PoP in different cities. A VPN service DIF on top of the Metro and Core DIFallocates resources to different VPN DIFs over the entire scope of the provider network.
15
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
3 Experiment 1: Management of multi-layer converged service pro-vider networks
3.1 Objectives
The main goal of experiment 1 is to demonstrate the difference between a (protocol) stack and aDIF. Traditionally, in the literature as well as in commercial environments, the network is seen asa stack or protocol stack, realised by all components in the network. Figure 8 shows an exampleof this view using the resource abstraction introduced in [2]. Each vertical in the figure showsa network component, for instance an end system, a border router, or an interior router. Eachpair of communication components must implement entire stacks that are compatible with eachother, where most components must implement the whole stack (left and right) or parts of thestack (middle) depending on what network functionality it provides. In the figure, the left and rightcomponent could be end systems and the middle component could be a switch or router.
Layer n
Layer n+1
Layer n-1
Figure 8: Management Points in a Protocol Stack Pattern
3.1.1 The Current Problem
The management of stacks is complex because it is necessary to coordinate vertical as well ashorizontal aspects, each separated per component and provided stack element. On the horizontalaspect, each part of a communication service (each stack element on the same layer in eachcomponent), needs to be configured individually. A reconfiguration often requires to reconfigure allstack elements of all components. This situation is even more complicated when we apply a real
16
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
word scenario, in which the ownership of the components varies. Any initial configuration and anyfurther reconfiguration must now be coordinated amongst all components and their respective owner(assuming that the owner has administrative permissions to perform the configuration action).
On the vertical, the configuration of each stack element must be configured in a way that thecomponent overall can provide the advertised and required service. Here, ownership is not theproblem. Instead the multi-vendor, multi-protocol, and multi-technology aspect of each individualstack layer provides for extensive complexity.
In a 5G converged service provider network, we must also consider a separation of technologyand ownership across the whole network, for components (here called nodes) as well as the“protocol zoo” dealing with various aspects of the network (and service) operation. Radio nodesform part of the radio network (RAN). Nodes aggregating radio access traffic and managing usersessions are found in the core network. Long distance communication and links to the publicInternet belong to the transport network. Each of those networks is often owned by different partsof a provider’s organisation.
This mix makes coordinated management of (protocol stacks) virtually impossible, mainlybecause the state space that needs to be coordinated is too large. The current solution is twofold.First, a large amount of standards is produced as normative source for management activities (theactivities themselves as well as for the underlying stack). Those standards provide interoperabilitybut limit flexibility. Second, management is often based on management models using managedobjects (with standardised access in form of a defined protocol and standardised information models,often in terms of a management information base). A large number of models exists and needs tobe considered for management activities. The figure 8 also shows the management points (blueinterfaces) and the internal management or configuration policies that need to be coordinated. Whilethe figure suggests a largely normalised environment, reality is much more heterogeneous.
3.1.2 Multi-layer Coordination
In RINA, the situation is very different. Using a common API (the IPC API) and building immutableinfrastructure used by each and every IPCP means that there are no longer an technologicaldifferences among different layers in the network. There are also no differences between verticalelements in a component, everything is an IPC process (IPCP).
Next, the concept of separation of mechanism and policy, and the definition of policies forall aspects of an IPCP (and thus inter-process communication), makes all available policies ex-plicit. There is no need anymore for separate management model interface definitions, everythingmanagement requires is already defined by the architecture.
Next, the concept of a DIF as a layer, and not a stack, with autonomic functions for layermanagement (as part of every IPCP) and with an explicit definition of the scope of the layer (thescope of the DIF, realised by its policies), provides for a very new concept with regard to multi-layermanagement.
Last but not least, the RIB in the IPCPs (and thus in the DIF) provides for the required
17
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
information base for all of the above. The RIB maintains all shared state of all IPCPs in a DIF. Thedefined processes also realise an automatic update of this shared state amongst all IPCPs in a DIF.This means that changes to a DIF’s configuration will propagate from one IPCP to another until theentire DIF is reconfigured.
3.1.3 The resulting Objectives
The objectives of experiment 1 are
1. to demonstrate that the simplification RINA provides (extensively discussed in [2] andsummarised above) lead to a significant simplification of the required management definitionsand implementations,
2. to demonstrate that, since the manager can use the same model with a few variations based ondifferent RINA policies (DIF policies) to configure multiple layers, coordinated managementcan evolve from a complex management task (often provided by workflows with associatedmanagement policies) into a strategy-oriented management task in which the managementactivities are solely described by management strategies (as discussed in [2]), and
3. to demonstrate that the manager can perform an adequate number of strategies at the sametime (single manager deployment), and
4. to demonstrate that the manager can be scaled up and scaled down in case a single managerdeployment is not able to handle the load of executed strategies
Multi-layer management can now evolve from stack management (vertical and horizontal)towards an actual coordination function in a multi-domain model (here the domains are DIFs).Figure 9 shows this new situation provided by RINA using the resource abstraction introduced in[2], including the management points (per domain element - IPCP, per domain - DIF, multi-domain,and for the management system on the right). Most of the control functions are realised in theIPCPs and the DIFs. DIF interaction is also provided by RINA. What is left for the coordination isto provide for a coordinated initial configuration, and later for possible re-configuration as part ofthe management activity (monitor and repair, see [2].
3.1.4 Scope: Configuration Management
Management activities cover a wide range of functional aspects. Since this experiment cannot coverall aspects, we will narrow the scope to a representative set of those aspects.
Network management activities are often called management functions. The functional scope ofthose management functions is commonly described in terms of Fault, Configuration, Accounting,Performance, and Security (FCAPS), also known as management areas [3]. On top of network
18
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Domainn
Domainn-1
Domainn+1
Mul�-Domain
Figure 9: Management Points in a Layered Domain Pattern
management (including layer management) system management functions are standardised tomanage a single system, see for instance ITU-T recommendations X.730 to X.799.
A management system, including the DMS, follows the common manager-agent paradigm,where a manager executes the management functions and a managed agent controls resources. Thecommunication between manager and agent is then the management protocol. Communication isrestricted to operations from manager to agent and notification from agent to manager. The agentuses a Management Information Base (MIB) as a standardised knowledge base of the resources itcontrols. The resources commonly provide a standardised interface to the agent.
Fault
Strategy
Strategy
Strategy
Strategy
CDAP
Opera�ons
No�fica�ons
Manager ManagementAgent
Distributed IPC Facility (DIF)
Border RouterInterior Router
Host Host
App A App B
DIF DIF
DIFDIF
Border Router
DIF
IPCPIPCP
IPCP
IPCP
IPCPIPCPIPCPIPCPIPCP
IPCP
IPCP IPCPIPCPIPCP
IPCP
Configura�on
Performance
Accoun�ng
Security
Figure 10: FCAPS, Strategies, and RINA Network
Figure 10 shows an example of the manager/agent paradigm in a RINA context. On the managerside, the FCAPS management areas are used to define the management functions. Strategies realisethose functions, so they are the management activities. The management protocol is CDAP. TheManagement Agent (MA) then controls a RINA network, i.e. a deployment of DIFs with IPC
19
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
processes. The MIB in RINA is superset of all the RINA Information Bases (RIB) of the controlledIPC processes. The common (standardised) management interface of the resources (DIFs, IPCprocesses) is the common IPC API.
As stated above, this experiment cannot cover all functional areas. For the FCAPS, the followingassumptions can be made to narrow the scope for the experiment:
• Security - the management of security aspects is covered in the ARCFIRE experiment 4.
• Accounting - while the collection of data for resource use can be realised. Since a DIF has awell-defined scope, such a collection can be done per DIF (and then per IPC process in theDIF) with the full understanding of the DIFs scope. Call Data Records (CDRs) or other baseinformation for accounting can then be produced. However, to demonstrate full AccountingManagement (AM), we would need to associate the usage data (e.g. CDRs) to user session,then individual users, and finally to the contract those users have with the service provider.This means building such a system requires a significant amount of resources outside theoriginal scope of the project yet with (very likely) little added value for the experiment.
• Performance - collection of performance data is very similar to collection of accountingdata. However, to demonstrate full Performance Management (PM) requires evaluating thecollected performance counters against the KPIs a network operator (or service provider) setfor its operation. Those goals differ from operator to operator and from provider to provider.They also depend on the optimisation strategy for a network (for instance optimise for voice,or data, or coverage, or against connection loss / call drop, etc.). There is little discussion inthe literature on common performance KPIs, beside some higher level goals such as optimalperformance. Since general performance tests of a RINA network have been done in otherprojects, performance management for this experiment will not add much value (besidesthe resource heavy and largely arbitrary design and implementation of several optimisationstrategies to test).
• Fault - faults are problems in the network that can be observed either directly (the resource andthen the Management Agent send an error in form of a notification) or indirectly (by observingthe behaviour of the network comparing it to the anticipated or required behaviour). Anexample of a directly observed fault is the loss of connectivity due to a hardware problem (portbroken, physical node down). Examples for an indirectly observed fault is congestion control(monitor the network for indicators of congestion) or the classic sleeping cell scenario (aradio node is alive, not down or broken, but does not accept any further traffic or connectionsfrom mobile equipment). While Fault Management (FM) is often seen as the most importantmanagement activity, it is very hard to define fault scenarios for a repeatable experimentationwith measurable KPIs. On one side, we can of course design an experiment in which anIPC process is deliberately broken, creating a fault, resulting in the DMS executing a faultmitigation strategy. On the other side, RINA is an autonomic network, that realises much
20
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
control (and management functionality, in the IPC process layer management) locally withoutnotifying the DMS. The actual possible fault scenarios in such an environment have not yetbeen studied in detail, thus making it very hard to design fault management scenarios.
• Configuration - every network operation starts with planning (design a network for givenrequirements) followed by deployment (of network nodes and other physical resources andsoftware components and other virtual resources). Once in operation, a network might(and usually does) require re-configuration. This requirement can be the result of a changein requirements (up to business goals set by the operator), faults in the network (faultswhich can be mitigated by re-configuration), a required shift in the operation due to changedtraffic (traffic behaviour or traffic mix impeding for instance required performance), theintroduction of new accounting measurements required for new accounting (and finallycharging) strategies, or changes for the security of the network operation (or parts of it).From this viewpoint, configuration management is the facilitator of all other functionalmanagement areas. Thus the main target for this experiment is Configuration Management(CM). Another reason to focus on CM is the current situation of support for deploying aRINA network. The RINA Demonstrator is the most efficient way to define a RINA networkand deploy it. The Demonstrator automates a large amount of the underlying configurationspecification and the deployment actions. In defining strategies for CM for this experiment,we can (i) benefit from the developed Demonstrator (specification requirements and workflowhave been developed, tested, and are in use for demonstrations for a long time) and (ii)provide a simpler, more dynamic, and more flexible solution than the demonstrator is today.A DMS with a set of CM strategies can supplement the Demonstrator and help to increasethe automated deployment of RINA networks. Adding reconfiguration (in the DMS) willfurther enhance demo capabilities.
3.1.5 Aspects, KPIs, Scenarios and Use Cases
Measuring a network management system for performance, cost, and quality is a very difficult task.All three aspects depend on many variables, e.g. network topology, network size, operational opti-misation strategies, available resources (compute, storage, network) for the network managementsystem, different deployment options (e.g. clustering) of the network management system, andmore. Beside measurable technical KPIs, a network management system has to support KPIs thatdepend on operator-specific goals and metrics, because a network management system will be usedin a specific environment of a particular operator, often a Network Operation Centre (NOC). SomeKPIs used are:
• network server availability
• server (NMS) to administration (NOC personal) ratio
21
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• network availability
• capacity of network path (to and from the NMS and in the managed network)
• Critical outage time (how long does it take the NMS to mitigate critical outages (i) in thenetwork and (ii) for itself)
• Frequency of monitoring checks (how often is the NMS required to pull monitoring informa-tion for its own operation)
• Number of affected users (NOC users) on an average basis, i.e. how many people are requiredto use the NMS to operate a network
• Indices of smoothness of network changes, service introduction, etc.
• Complexity of network management operations, e.g. how many different actions need to beinvoked to solve a problem
• Performance of network management operations, e.g. how many similar actions are need tobe invoked solve a problem
• Workflow complexity, e.g. how complex are workflows for problem solving strategies
• Touchpoints, e.g. how many touchpoints are required to realise a network managementoperation (zero-touch here means none, as the ultimate goal and key indicator for a highdegree of automation in the network management system)
Detailed statistics and performance of Operation Support Systems (OSS) used by mobileoperators cannot be obtained. This information is often considered to be privileged by operators aswell as OSS vendors. The same situation applies to metrics on NMS ease of use, involved personaland other costs, and quality.
The closet equivalent to the ARCFIRE DMS in the real world are SNMP based managementsystem for mobile, core, and transport networks as well as for LAN/MAN deployments. However,as [4] states, the performance of SNMP (and related NMSs) has never been fully and properlystudied. There is no commonly agreed measurement methodology to assess network management(and SNMP) performance). In addition, similar to OSS, little is known about SNMP (and SNMP-based NMS) usage patterns in operational networks, which impedes design of realistic scenariosfor an NMS analysis.
The search for relevant, measurable, and comparable KPIs for this experiment is furthercomplicated by the radical new value propositions of a RINA network. For the first time we have afully autonomic network available, including explicit specifications of the scope of layers (DIFs),automatic mechanisms for sharing state between layer elements (the IPC process with their RIBinside a DIF), fully policy-based local control (data transfer and data transfer control), and the
22
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
availability of a complete set of layer management functions (the layer management functions insidean IPC process). Multi-layer (multi-DIF) coordination now becomes a very different and simplertask (compared to current OSS and NMS systems). Even comparing it to cross-layer optimisationis difficult, since this optimisation assumes a role of control over the layers while the ARCFIREDMS operates in terms of monitoring and repair. The starting point of management activities isvery different, making comparisons extremely difficult (or arbitrary).
To provide a consistent, measurable, and comparable set of performance indicators for the DMSwe will focus on three aspects:
• Speed: the speed of management operations can be measured by the number of collected,required, and adjusted variables (policies and configurations of DIFs); the number of notifica-tions for a management activity; and experienced delays (in the management activity).
• Cost: can be measured by CPU usage (or wider usage of compute resources in a cloudenvironment), memory usage (or the usage of storage resources in a wider cloud environment),and bandwidth usage (how much overhead in required bandwidth does management create).
• Quality: quality can be measured in a spatial-temporal context. Spatial here refers todifferences in particular variables (configuration and policies) between the DMS and theRINA networks. Temporal refers to errors due to temporal aspects, such as monitoringroundtrip times. Loss can be added to evaluate if information is lost in transport.
For these three aspects, we can now look into relevant KPIs. We have identified the followinggeneral KPIs for this experiment:
• Speed: the speed of management strategies
• Scale: the required scale of the DMS for speedy operation
• Time and cost for scale-in and scale-out: how timely can the DMS be scaled out (extended)in case of cascading event storms, how costly is this change of scale
• Touch: how many touches are required for management strategies to succeed (this can beused as an indicator for cost since it determines the extent of human input required for anoperation)
• Complexity: how complex are the used management strategies, i.e. how many differentoperations do they require (internally to make decisions and externally to realise them)
• Degree of automation: using the touch KPI we can estimate the degree of automation theDMS will provide in a network operation
23
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Aspects and general KPIs need to be measured in experiments. Those experiments can beclassified as a combination of a scenario with a use case. There are many possible scenarios anduse cases to select from. The following scenarios are considered:
• No network: this scenario can be used to benchmark the DMS and all developed strategies.Strategy triggers (and responses) need to be simulated but actions are not performed. Thefocus is on the DMS performance only. The DMS configuration is shown in Figure 11.
• Small (minimal) RINA network: a simple deployment of two hosts, each connected to adedicated border router, both border routers connected to the same interior router. The DMSconfiguration is shown in Figure 12. The RINA network setup is shown in Figure 13.
• Medium RINA network: a medium size RINA network as for instance used in the renumber-ing demonstration (30 nodes to show a network on European scale as used in experiment 3,cf Figure 20) or the data center demonstration (38 nodes for a medium size data center usinga spine-leaf configuration). The DMS configuration is shown in Figure 12.
• Large RINA network: a large RINA network like the network used in the experimentresembling the AT&T transport network for managed services (as used in experiment 3, cfFigure 21) The DMS configuration is shown in Figure 12.
DMS
DMSStrategy Engine
RIB
ManagementShell / GUI
OperatorOSS/NMS and
OpenNMS
Message Bus (W3C WebSocket)
DMSStrategy Engine
Figure 11: DMS: Stand-alone System for Benchmark Experiments
Each these scenarios can be combined with each of the following use cases:
• Deploy network from zero: no RINA network exists, the DMS is triggered by an operatormechanism and deploys a complete RINA network.
• Deploy a new service (DIF) in existing network: A RINA network exists, the DMS managesit, now add services and applications to the network by creating a new DIF (service) anddeploying applications.
24
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
RINA
DMS
ManagedResource
(RINA)
ManagementAgent(MA)
DMSStrategy Engine
API Calls, etc.CDAP
RIB RIB RIBSynchroniza�on Synchroniza�on
ManagementShell / GUI
OperatorOSS/NMS and
OpenNMS
Message Bus (W3C WebSocket) CDAPConnector
DMSStrategy Engine
Figure 12: DMS: Full System for Experiments with a RINA Network
Distributed IPC Facility (DIF)
Border RouterInterior Router
Host Host
App A App B
DIF DIF
DIFDIF
Border Router
DIF
IPCPIPCP
IPCP
IPCP
IPCPIPCPIPCPIPCPIPCP
IPCP
IPCP IPCPIPCPIPCP
IPCP
Figure 13: Experiment 1: Minimal RINA System for simple Experiments
25
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• Deploy network and service from zero: Deploy the network nodes, all DIFs, and an applica-tion.
• Add a new node to network: how many touch points are required to add a new node, e.g. is itpossible to create a zero touch strategy for adding new nodes?
The resulting 4-tuple for this experiment then is aspects, general KPIs, scenarios, and use cases.For concrete experiments we should define concrete KPIs and design the required strategies. It isalso clear that a separation into sub-experiments is required. This separation can be done usingthe scenarios or the use cases. We have decided to take the use cases as key separator and run allscenarios in each use case.
The following sub sections detail the resulting 4 sub experiments. Each sub experimentaddresses one use case for all scenarios using all KPIs covering all aspects. Therefore, the subexperiment description is in parts repetitive. We chose to leave the repetition in this documentation,because different teams may only work a subset of the sub experiments.
3.2 Experiment 1-1: Deploy Network from Zero
3.2.1 Objectives
This experiment assesses the DMS’s capability to build a network from scratch using the specifica-tion of an operator’s network planning, e.g. topology and required network services as input. At thestart, there is no network, i.e. no network node is up and running. The first step is to start the firstnetwork node. Each following step adds a new network node according to the given topology.
The configurations of all nodes is handled by the DMS. A single strategy should be specified,which takes a topology (and all required information for specific nodes, e.g. required services andDIFs) and once triggered, builds the network step by step (i.e. node by node).
The experiment is finalised by a second strategy, which deploys a number of tests in the newlybuild RINA network to evaluate the correctness of the configuration. Those tests can be active(deploy a test service and evaluate correctness) or passive (analyse the configuration of each nodeas represented in the RIB against the given topology).
The strategy which builds the network can be configured for the three RINA network scenarios.The experiment can then be run for the benchmark scenario and all three RINA network scenarios.We do not anticipate different strategies for the different scenarios.
The initial trigger for building the RINA network comes from the operator’s OSS/NMS. Thiswill be simulated by a skeleton OSS/NMS trigger application in the DMS. Evaluation of the builtnetwork in the benchmark scenario can only be performed passively.
3.2.2 Metrics and KPIs
This experiment uses the KPIs introduced in section 3.1.5 in terms of speed, scale, time and cost,touch, complexity (of the strategy), and degree of automation.
26
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
They are further specialised for this experiment as shows in Table 1. This experiment shouldquantify:
• the speed a node is added to the network (the complete transaction from strategy trigger towhen the node is up and running)
• the speed a whole RINA network is built (the complete transaction from strategy triggerto when the network is up and running), very likely determined by the number of similaroperations the management strategy must realise
• the scale of the required DMS (one or more executors, possibility of parallelising the actionsfor adding nodes)
• time and cost to scale the DMS up or down (e.g. per RINA network scenario or when runningdifferent scenarios in sequence)
• the complexity of the management strategy, i.e. the number of different operations thestrategy must realise
• touches and degree of automation, evaluated as the number of touches required to build acomplete RINA network
3.2.3 Software
This experiment will require several software items for its execution, summarised in Table 2.
• IRATI RINA implementation. It will be used to implement multiple RINA-enabled systemsand run a variety of DIFs according to the experiment configuration.
• rina-echo-time. The rina-echo-time application will be used to measure the latency andpacket loss perceived by a flow user while the experiment is running.
• rina-tgen. The rina-tgen application will be used to create traffic while the experiment isrunning.
• Demonstrator. The Demonstrator will be used to provide as a base tool for the configurationof the RINA network. Once an associated strategy is developed, it should be possible toremove the Demonstrator from the experiment.
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
27
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
KPI Metric Current state of the art ARCFIRE Objective
E1-1.1 Speedy node add ms or s Node creation and config-uration is often realised byrather complex workflowsthat can take several minutesup to an hour (for automatedcreation)
the strategy should run insub-second speed for singlenode creation
E1-1.2 Speedy networkcreation
ms or s or min Creation and configurationof a whole network can,depending on the networkcomplexity, take severalhours to several weeks
Even the large RINA net-work should be created inminutes, rather than hours
E1-1.3 DMS scale Number of strategy execu-tors
Multiple workflows executedin parallel (or realised byteams working in parallel)
Ideally only 1 executorrequired (sufficient speed),more (with parallelisedexecution) if speed KPIcannot be met
E1-1.4 DMS Scale Out s and CPU usage Scaling out a managementsystem that is in operationis often impossible or ex-tremely costly (takes hours,requires multiple touches, isprocessing intensive)
Scaling out the DMS shouldbe done in seconds, with 1or zero touch, and minimalCPU costs (for the scalingitself)
E1-1.5 Strategy Complexity number of different opera-tions required
Currently, CM operationscreate large sets of CMprofiles, one per node
From early experiments, weestimate 4-5 different oper-ations per node replicatedfor each node to create thenetwork
E1-1.6 Touches / Degree ofAutomation
number of touches Beside the (not counted)original touch (installationof hardware node or triggerfor software installation),adding a node often requiresmultiple additional touchesfor configuration. Early pro-totypes for VNF deploymentcan operate on a minimum(potentially zero) touchbasis.
Ideally zero touch
Table 1: KPIs for experiment 1-1
28
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
DMS Management system with strategy executor, OSS/NMS trigger, management shell,event visualiser
TBD
Table 2: Software required for experiment 1-1
• DMS The DMS is the core system under evaluation in the experiment. It will be deployedin the required configuration (see experiment scenarios). The development of an associatedstrategy for the experiment is important, which will be deployed in the DMS and triggered bythe experiment. An additional small application will be used in the DMS to trigger strategyexecution, in particular an OSS/NMS trigger simulator for the benchmark scenario.
3.2.4 Testbeds
The Virtual Wall will be the main testbed of reference as reported in D4.2 [9]. The Virtual Wallcan provide access to bare metal hardware to run the RINA implementation and the applicationsrequired for the experiment, providing access to an adequate number of resources.
Initially the jFed experimenter GUI will be used to reserve and setup the resources in theFED4FIRE+ testbeds. Once it becomes available, the experimentation and measurement frameworkunder development in WP3 will provide a more automated front-end for jFed as well as other IRATIdeployment and configuration tools such as the Demonstrator.
The DMS will be deployed along with the IRATI Demonstrator.In summary, this experiment may use the following testbeds from the selection made in D4.2
[9]:
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out experiment 1-1
3.2.5 Experiment scenarios
The experiment will be run in four different scenarios.No RINA Network: This scenario uses triggers from the operators OSS/NMS (here simulated
by a skeleton application) to benchmark the DMS strategy execution. All developed strategies willbe tested for speed and scale. The DMS will be a stand-alone DMS as shown in Figure 11.
29
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
MS1 M16 DMS Software with Strategy Executor, OSS/NMS trigger, MA/Demonstrator Integration
MS2 M17 Strategy for experiment defined and tested
MS3 M21 reference experiment with measurements on LMI reference server for direct comparison for all scenar-ios
MS4 M22 continuous experiments for scenario 1 (benchmarking)
MS5 M24 continuous experiments for scenario 2 (minimal RINA network)
MS6 M26 continuous experiments for scenario 3 (medium size RINA network)
MS7 M28 continuous experiments for scenario 4 (large size RINA network)
Table 3: Milestones for experiment 1-1
Minimum RINA Network: This scenario uses the minimum RINA network (2 hosts, 2 borderrouters, 1 interior router, cf. Figure 13) with an associated strategy for the experiment. The DMSwill be a full configuration as shown in Figure 12.
Medium Size RINA Network: This scenario uses a medium size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 20 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
Large Size RINA Network: This scenario uses a large size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 21 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
3.2.6 Planning
The milestones for the design and specification of experiment 1-1 are shown in Table 3.
3.3 Experiment 1-2: Deploy new Service in Existing Network
3.3.1 Objectives
This experiment assesses the DMS’s capability to add a new network service to an existing RINAnetwork. The starting point is a RINA network, which can be built using the strategies of experiment1-1. Once the network is built, a new network service (a new DIF) is injected into the existingnetwork. The configuration and scope of this DIF is provided by the operator, e.g. a planning task.This experiment has only 1 step: add a new DIF to an existing network.
The configuration and scope of the new DIF is handled by the DMS. A single strategy shouldbe specified, which takes the operator’s planning specification and, once triggered, creates the DIF.
The experiment is finalised by a second strategy, which deploys a number of tests in the newlycreated DIF to evaluate the correctness of its configuration and scope. Those tests can be active
30
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
(deploy a test service and evaluate correctness) or passive (analyse the configuration of the DIFagainst the given planning specifications).
The strategy which adds a new DIF can be configured for the three RINA network scenarios.The experiment can then be run for the benchmark scenario and all three RINA network scenarios.We do not anticipate different strategies for the different scenarios.
The initial trigger for adding a new DIF comes from the operator’s OSS/NMS. This will besimulated by a skeleton OSS/NMS trigger application in the DMS. Evaluation of added DIF in thebenchmark scenario can only be performed passively.
3.3.2 Metrics and KPIs
This experiment uses the KPIs introduced in section 3.1.5 in terms of speed, touch, complexity (ofthe strategy), and degree of automation.
They are further specialised for this experiment as shows in Table 4. This experiment shouldquantify:
• the speed a DIF is added to the network (the complete transaction from strategy trigger towhen the DIF is up and running)
• the complexity of the management strategy, i.e. the number of different operations thestrategy must realise
• touches and degree of automation, evaluated as the number of touches required add a newDIF
KPI Metric Current state of the art ARCFIRE Objective
E1-2.1 Speedy DIF add ms or s Service creation and config-uration is often realised byrather complex workflowsthat can take several min-utes or hours (for automatedcreation)
the strategy should run insub-second speed
E1-2.2 Strategy Complexity number of different opera-tions required
Currently, CM operationscreate large sets of CMprofiles, one per node
1 operation replicated percreation of IPC process onparticipating nodes
E1-2.3 Touches / Degree ofAutomation
number of touches Creation of new services inthe network is an extremelycomplex and complicatedprocess, with multiple teamsand touches
Ideally zero touch
Table 4: KPIs for experiment 1-2
31
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
DMS Management system with strategy executor, OSS/NMS trigger, management shell,event visualiser
TBD
Table 5: Software required for experiment 1-2
3.3.3 Software
This experiment will require several software items for its execution, summarised in Table 5.
• IRATI RINA implementation. It will be used to implement multiple RINA-enabled systemsand run a variety of DIFs according to the experiment configuration.
• rina-echo-time. The rina-echo-time application will be used to measure the latency andpacket loss perceived by a flow user while the experiment is running.
• rina-tgen. The rina-tgen application will be used to create traffic while the experiment isrunning.
• Demonstrator. The Demonstrator will be used to provide as a base tool for the configurationof the RINA network. Once an associated strategy is developed, it should be possible toremove the Demonstrator from the experiment.
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
• DMS The DMS is the core system under evaluation in the experiment. It will be deployedin the required configuration (see experiment scenarios). The development of an associatedstrategy for the experiment is important, which will be deployed in the DMS and triggered bythe experiment. An additional small application will be used in the DMS to trigger strategyexecution, in particular an OSS/NMS trigger simulator for the benchmark scenario.
3.3.4 Testbeds
The Virtual Wall will be the main testbed of reference as reported in D4.2 [9]. The Virtual Wallcan provide access to bare metal hardware to run the RINA implementation and the applicationsrequired for the experiment, providing access to an adequate number of resources.
32
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Initially the jFed experimenter GUI will be used to reserve and setup the resources in theFED4FIRE+ testbeds. Once it becomes available, the experimentation and measurement frameworkunder development in WP3 will provide a more automated front-end for jFed as well as other IRATIdeployment and configuration tools such as the Demonstrator.
The DMS will be deployed along with the IRATI Demonstrator.In summary, this experiment may use the following testbeds from the selection made in D4.2
[9]:
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out experiment 1-2
3.3.5 Experiment scenarios
The experiment will be run in four different scenarios.No RINA Network: This scenario uses triggers from the operators OSS/NMS (here simulated
by a skeleton application) to benchmark the DMS strategy execution. All developed strategies willbe tested for speed and scale. The DMS will be a stand-alone DMS as shown in Figure 11.
Minimum RINA Network: This scenario uses the minimum RINA network (2 hosts, 2 borderrouters, 1 interior router, cf. Figure 13) with an associated strategy for the experiment. The DMSwill be a full configuration as shown in Figure 12.
Medium Size RINA Network: This scenario uses a medium size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 20 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
Large Size RINA Network: This scenario uses a large size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 21 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
3.3.6 Planning
The milestones for the design and specification of experiment 1-2 are shown in Table 6.
3.4 Experiment 1-3: Deploy Network and Service from Zero
3.4.1 Objectives
This experiment assesses the DMS’s capability to build a new RINA network and add all requirednetwork services, application services, and applications to it. The starting point for this experimentis a topology based on the operator’s topology for the network, the requirements for networkand application services based on operator’s specifications, and a set of applications that must besupported by the network as defined by the operator.
33
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
MS1 M16 DMS Software with Strategy Executor, OSS/NMS trigger, MA/Demonstrator Integration
MS2 M18 Strategy for experiment defined and tested
MS3 M21 reference experiment with measurements on LMI reference server for direct comparison for all scenar-ios
MS4 M22 continuous experiments for scenario 1 (benchmarking)
MS5 M24 continuous experiments for scenario 2 (minimal RINA network)
MS6 M26 continuous experiments for scenario 3 (medium size RINA network)
MS7 M28 continuous experiments for scenario 4 (large size RINA network)
Table 6: Milestones for experiment 1-2
At the start, there is no network, i.e. no network node is up and running. The first step is tostart the first network node. Each following step adds a new network node according to the giventopology. Then, the required services are added (e.g. related DIFs created). Finally, all requiredinfrastructure for the applications are deployed in the network. For the applications, we will use thestandard IRATI applications rina-echo-time and rina-tgen.
All configurations are handled by the DMS. A single strategy should be specified that createsthe network, adds all required services, and finally deploys the application infrastructure.
The experiment is finalised by a second strategy, which deploys a number of tests in the newlycreated network to evaluate the correctness of its configuration and scope. Those tests can be active(deploy a test application and evaluate correctness) or passive (analyse the network’s configurationagainst the given operator’s specifications).
The creating strategy can be configured for the three RINA network scenarios. The experimentcan then be run for the benchmark scenario and all three RINA network scenarios. We do notanticipate different strategies for the different scenarios.
The initial trigger comes from the operator’s OSS/NMS. This will be simulated by a skeletonOSS/NMS trigger application in the DMS. Evaluation the network in the benchmark scenario canonly be performed passively.
3.4.2 Metrics and KPIs
This experiment uses the KPIs introduced in section 3.1.5 in terms of speed, scale, time and cost,touch, complexity (of the strategy), and degree of automation.
They are further specialised for this experiment as shows in Table 7. This experiment shouldquantify:
• the speed a node is added to the network (the complete transaction from strategy trigger towhen the node is up and running)
34
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• the speed a whole RINA network is built (the complete transaction from strategy trigger tonetwork up and running), very likely determined by the number of similar operations themanagement strategy must realise
• the speed a DIF is added to the network (the complete transaction from strategy trigger towhen the DIF is up and running)
• the scale of the required DMS (one or more executors, possibility of parallelising the actionsfor adding nodes)
• time and cost to scale the DMS up or down (e.g. per RINA network scenario or when runningdifferent scenarios in sequence)
• the complexity of the management strategy, i.e. the number of different operations thestrategy must realise
• touches and degree of automation, evaluated as the the number of touches required to build acomplete RINA network and deploy a new DIF
3.4.3 Software
This experiment will require several software items for its execution, summarised in Table 8.
• IRATI RINA implementation. It will be used to implement multiple RINA-enabled systemsand run a variety of DIFs according to the experiment configuration.
• rina-echo-time. The rina-echo-time application will be used to measure the latency andpacket loss perceived by a flow user while the experiment is running.
• rina-tgen. The rina-tgen application will be used to create traffic while the experiment isrunning.
• Demonstrator. The Demonstrator will be used to provide as a base tool for the configurationof the RINA network. Once an associated strategy is developed, it should be possible toremove the Demonstrator from the experiment.
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
• DMS The DMS is the core system under evaluation in the experiment. It will be deployedin the required configuration (see experiment scenarios). The development of an associatedstrategy for the experiment is important, which will be deployed in the DMS and triggered bythe experiment. An additional small application will be used in the DMS to trigger strategyexecution, in particular an OSS/NMS trigger simulator for the benchmark scenario.
35
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
KPI Metric Current state of the art ARCFIRE Objective
E1-3.1 Speedy node add ms or s Node creation and config-uration is often realised byrather complex workflowsthat can take several minutesup to an hour (for automatedcreation)
the strategy should run insub-second speed for singlenode creation
E1-3.2 Speedy networkcreation
ms or s or min Creation and configurationof a whole network can,depending on the networkcomplexity, take severalhours to several weeks
Even the large RINA net-work should be created inminutes, rather than hours
E1-3.3 Speedy DIF add ms or s Service creation and config-uration is often realised byrather complex workflowsthat can take several minutesor an hours (for automatedcreation)
the strategy should run insub-second speed
E1-3.4 DMS scale Number of strategy execu-tors
Multiple workflows executedin parallel (or realised byteams working in parallel)
Ideally only 1 executorrequired (sufficient speed),more (with parallelisedexecution) if speed KPIcannot be met
E1-3.5 DMS Scale Out s and CPU usage Scaling out a managementsystem that is in operationis often impossible or ex-tremely costly (takes hours,requires multiple touches, isprocessing intensive)
Scaling out the DMS shouldbe done in seconds, with 1or zero touch, and minimalCPU costs (for the scalingitself)
E1-3.6 Strategy Complexity number of different opera-tions required
Currently, CM operationscreate large sets of CMprofiles, one per node andservice
From early experiments, weestimate 4-5 different oper-ations per node replicatedfor each node to create thenetwork plus deploying therequired network services
E1-3.7 Touches / Degree ofAutomation
number of touches Adding nodes and servicesoften requires multipletouches, realised by multipleteams. Early prototypes forVNF deployment can operateon a minimum (potentiallyzero) basis.
Ideally zero touch
Table 7: KPIs for experiment 1-3
36
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
DMS Management system with strategy executor, OSS/NMS trigger, management shell,event visualiser
TBD
Table 8: Software required for experiment 1-3
3.4.4 Testbeds
The Virtual Wall will be the main testbed of reference as reported in D4.2 [9]. The Virtual Wallcan provide access to bare metal hardware to run the RINA implementation and the applicationsrequired for the experiment, providing access to an adequate number of resources.
Initially the jFed experimenter GUI will be used to reserve and setup the resources in theFED4FIRE+ testbeds. Once it becomes available, the experimentation and measurement frameworkunder development in WP3 will provide a more automated front-end for jFed as well as other IRATIdeployment and configuration tools such as the Demonstrator.
The DMS will be deployed along the IRATI Demonstrator.In summary, this experiment may use the following testbeds from the selection made in D4.2
[9]:
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out experiment 1-3
3.4.5 Experiment scenarios
The experiment will be run in four different scenarios.No RINA Network: This scenario uses triggers from the operators OSS/NMS (here simulated
by a skeleton application) to benchmark the DMS strategy execution. All developed strategies willbe tested for speed and scale. The DMS will be a stand-alone DMS as shown in Figure 11.
Minimum RINA Network: This scenario uses the minimum RINA network (2 hosts, 2 borderrouters, 1 interior router, cf. Figure 13) with an associated strategy for the experiment. The DMSwill be a full configuration as shown in Figure 12.
Medium Size RINA Network: This scenario uses a medium size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 20 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
37
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Large Size RINA Network: This scenario uses a large size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 21 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
3.4.6 Planning
The milestones for the design and specification of experiment 1-3 are shown in Table 9.
Milestone Month Description
MS1 M16 DMS Software with Strategy Executor, OSS/NMS trigger, MA/Demonstrator Integration
MS2 M19 Strategy for experiment defined and tested
MS3 M21 reference experiment with measurements on LMI reference server for direct comparison for all scenar-ios
MS4 M22 continuous experiments for scenario 1 (benchmarking)
MS5 M24 continuous experiments for scenario 2 (minimal RINA network)
MS6 M26 continuous experiments for scenario 3 (medium size RINA network)
MS7 M28 continuous experiments for scenario 4 (large size RINA network)
Table 9: Milestones for experiment 1-3
3.5 Experiment 1-4: Add new Network Node - Zero touch
3.5.1 Objectives
This experiment assesses how many touch points are required to add a new node to an existingRINA network. We consider any required manual action or necessary interaction of the DMS withan administrator as a touch point. Ideally, the DMS can add a new node with zero touches, i.e. fullyautomated. It is important for this experiment to establish how close the DMS can be to a zerotouch management system and what information is required to achieve that. Once described, it willalso be of value to minimise the information required.
The starting point for this experiment is a network of at least 1 node, since the first node mightrequire more configuration than any following node. The experiment can then run for every nodedescribed in a given topology. Ideally, the DMS can add all new nodes automatically.
All configurations are handled by the DMS. A single strategy should be specified that addsa new node to the network. After adding a new node, a second strategy should be triggered toevaluate the correct configuration of the newly added node against the given topology.
The initial trigger comes from the operator’s OSS/NMS. This will be simulated by a skeletonOSS/NMS trigger application in the DMS. Evaluation of the network in the benchmark scenariocan only be performed passively.
38
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
3.5.2 Metrics and KPIs
This experiment uses the KPIs introduced in section 3.1.5 in terms of touch and complexity (of thestrategy).
They are further specialised for this experiment as shows in Table 10. This experiment shouldquantify:
• touches and degree of automation as in the number of touches required to build a completeRINA network and deploy a new DIF
• the complexity of the management strategy, i.e. the number of different operations thestrategy must realise
KPI Metric Current state of the art ARCFIRE Objective
E1-4.1 Touches / Degree ofAutomation
number of touches Beside the (not counted)original touch (installation ofhardware node or trigger forsoftware installation), addinga node often requires multi-ple touches for configuration.Early prototypes for VNFdeployment can operate ona minimum (potentially 0)touch basis.
zero touch
E1-4.1 Strategy Complexity number of different opera-tions required
Currently, CM operationscreate large sets of CMprofiles per node
From early experiments, weare estimate 4-5 differentoperations per node
Table 10: KPIs for experiment 1-4
3.5.3 Software
This experiment will require several software items for its execution, summarised in Table 11.
• IRATI RINA implementation. It will be used to implement multiple RINA-enabled systemsand run a variety of DIFs according to the experiment configuration.
• rina-echo-time. The rina-echo-time application will be used to measure the latency andpacket loss perceived by a flow user while the experiment is running.
• rina-tgen. The rina-tgen application will be used to create traffic while the experiment isrunning.
39
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software Description License
IRATI [5] RINA Implementation for Linux O/S GPL and LGPL
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
DMS Management system with strategy executor, OSS/NMS trigger, management shell,event visualiser
TBD
Table 11: Software required for experiment 1-4
• Demonstrator. The Demonstrator will be used to provide as a base tool for the configurationof the RINA network. Once an associated strategy is developed, it should be possible toremove the Demonstrator from the experiment.
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
• DMS The DMS is the core system under evaluation in the experiment. It will be deployedin the required configuration (see experiment scenarios). The development of an associatedstrategy for the experiment is important, which will be deployed in the DMS and triggered bythe experiment. An additional small application will be used in the DMS to trigger strategyexecution, in particular an OSS/NMS trigger simulator for the benchmark scenario.
3.5.4 Testbeds
The Virtual Wall will be the main testbed of reference as reported in D4.2 [9]. The Virtual Wallcan provide access to bare metal hardware to run the RINA implementation and the applicationsrequired for the experiment, providing access to an adequate number of resources.
Initially the jFed experimenter GUI will be used to reserve and setup the resources in theFED4FIRE+ testbeds. Once it becomes available, the experimentation and measurement frameworkunder development in WP3 will provide a more automated front-end for jFed as well as other IRATIdeployment and configuration tools such as the Demonstrator.
The DMS will be deployed along with the IRATI Demonstrator.In summary, this experiment may use the following testbeds from the selection made in D4.2
[9]:
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out experiment 1-4
40
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
3.5.5 Experiment scenarios
The experiment will be run in four different scenarios.No RINA Network: This scenario uses triggers from the operators OSS/NMS (here simulated
by a skeleton application) to benchmark the DMS strategy execution. All developed strategies willbe tested for speed and scale. The DMS will be a stand-alone DMS as shown in Figure 11.
Minimum RINA Network: This scenario uses the minimum RINA network (2 hosts, 2 borderrouters, 1 interior router, cf. Figure 13) with an associated strategy for the experiment. The DMSwill be a full configuration as shown in Figure 12.
Medium Size RINA Network: This scenario uses a medium size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 20 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
Large Size RINA Network: This scenario uses a large size RINA network, for instance theEuropean network used in experiment 3 as shown in Figure 21 and an associated strategy for theexperiment. The DMS will be a full configuration as shown in Figure 12.
3.5.6 Planning
The milestones for the design and specification of experiment 1-4 are shown in Table 12.
Milestone Month Description
MS1 M16 DMS Software with Strategy Executor, OSS/NMS trigger, MA/Demonstrator Integration
MS2 M20 Strategy for experiment defined and tested
MS3 M21 reference experiment with measurements on LMI reference server for direct comparison for all scenar-ios
MS4 M22 continuous experiments for scenario 1 (benchmarking)
MS5 M24 continuous experiments for scenario 2 (minimal RINA network)
MS6 M26 continuous experiments for scenario 3 (medium size RINA network)
MS7 M28 continuous experiments for scenario 4 (large size RINA network)
Table 12: Milestones for experiment 1-4
3.6 Summary
In this section we have detailed the ARCFIRE experiment 1 focusing on multi-layer coordinationin a converged service provider network. We have defined the main objective, the current problem,and the solution that a multi-layer approach should provide in the context of RINA. We havealso detailed why we are focusing on configuration management. The experiment then is basedon a 4-tuple of aspects, KPIs, scenarios, and use cases. This tuple lead to the definition of 4
41
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
sub-experiments covering the four described scenarios, providing quantitative results for all fouruse cases, specifying individual KPIs and addressing all described aspects.
42
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
4 Experiment 2: Deploying resilient, virtualised services over hetero-geneous physical media
4.1 Introduction
Experiment 2 will explore the differences between RINA and TCP/IP when deploying resilient,virtualised services over heterogeneous physical media. The scenario considered here is mobileusers in different parts of Europe connecting over LTE/5G or Wi-Fi access technology. In depthdescriptions of LTE Advanced and WiFi are provided in [1]. Some analytical work has alreadybeen performed on mobility in RINA [10], which focused on comparing RINA to LISP and MobileIP. ARCFIRE experimentation will take a big leap by comparing a RINA 5G solution to the stateof the art in LTE Advanced protocols. Considerations for a 5G deployment based on RINA areprovided in Section 4 of D2.2 [2]. From these considerations, we extract the following requirementsfor an efficient RINA 5G deployment that will be taken as baseline for experiment 2 of ARCFIRE:
• Removal of anchor points: LTE uses either the P-GW or S-GW as anchor points andestablishes tunnels to the UE. Trying to mimic the use of anchor points in RINA wouldincrease complexity and reduce its efficiency. Rather, RINA points to a routing-baseddistributed management solution [11] to be implemented.
• Enhanced capability at the eNodeB: With the removal of the anchor point, the only locationleft that has the opportunity to correctly measure network usage by the end users is directly atthe base station. This means that service mobility as provided by an eNB and an associatedeNodeB Cloud (eNBC) will be mandatory.
4.2 Objectives
RINA’s simple architectural solution to multi-homing and mobility will be evaluated experimentallyagainst the current state-of-the-art of LTE and Wi-Fi networks. The most important aspectsconsidered are reducing overhead between the end user and the base station, the total signallingoverhead in the network, handover duration and packet loss during handovers. We will also evaluatethe impact on the core network, such as routing table sizes.
The key objectives are to find where RINA will perform better or worse than the custom-builtLTE solution. An observation is that the generality of the RINA architecture leads to the overallexpectation that the benefits will increase the further we move up the network hierarchy (i.e awayfrom physical layer constraints) the larger we consider the network, in terms of end user applications,deployed routers, management domains (think Autonomous Systems), and so on.
From this general observation, areas where we expect RINA to outperform 5G proposals builton SDN solutions are a general reduction of complexity, reducing the number of elements necessaryto operate the network to the same level of efficiency and performance. Replacing tunnels by a
43
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
routed solution removes a complex element (the MME) from the equation. A key objective will beto quantify this by measuring network characteristics across the board.
Another area where RINA should perform very well is reliability. Using a routed solution insmall DIFs, we can tailor routing updates to react very fast to changes in network connectivity.Whatevercast naming will also simplify managing handovers and mitigate connection interruptionsthat cannot be handled within a DIF (when the graph becomes separated into different components).Combined with the application addressing, we expect measurable decreases in connection downtimein the presence of averse network conditions. This work has already been investigated in thePRISTINE project at a smaller scale [12]. With the improved software frameworks that aredeveloped by ARCFIRE, we will be able to provide results in much more detailed scenarios.
Sticking points for a RINA deployment will be where hardware constraints will be felt themost, close to the transmission equipment. LTE has a very efficient solution for reducing overheadbetween the eNodeB and the UE in the RObust Header Compression (ROHC) technology used byPDCP, replacing L3 and L4 headers (IP/UDP/RTP) by a very small token, reducing 40 (or even60 bytes in case of IPv6) to 2-3 bytes [13]. RINA’s layered structure, with encapsulation at eachlayer to maintain transparency may introduce additional overhead at this point. As part of thisexperiment, we will investigate policies for header compression to reduce the Protocol ControlInformation in the PDUs on the eNodeB-UE link.
4.3 Experiment scenario
Experiment 2 considers mobile end users accessing a variety of services. There will be 4 servicesavailable in the experiments (Web, Voice, Gaming and File Transfer). These users will reside acrossEurope, interconnected over a single core network. A scenario involving multiple core networks isinvestigated in experiment 3. For the LTE based experiments, we will use the nginx web-server, theMidori web-browser, VLC for the video service, ioquake for the gaming service and Filezilla forthe file transfer. For the RINA experiments, ports are already available for nginx (the port will bereleased as open-source in the next month) and ioquake3 [14]. The other services will be validatedby comparing statistics gathered with the RINA tools like rina-tgen and rinaperf (described in [15],Section 3.1), which can be run by means of the Rumba framework. If additional RINA ports shouldbecome available (e.g. a browser of a video streaming server/client), they will be adopted in theexperiments as well.
In the mobile edge, we will investigate three access technologies: Fixed (Gb Ethernet), LTEand Wi-Fi and consider handovers within the same access technology and cross-technology andwithin an between network operators. This includes handovers between base stations of the sameoperator (X2 and S11 interface) and between different operators.
In the metro and core network, we will build on 1G and 10G Ethernet links for some bare-metalmeasurements. For scaled up experiments, we will use virtual machines in GENI and iLab.t(interconnected wherever possible) to perform experiments that are larger in terms of network hosts,but less demanding in terms of bandwidth usage. Such experiments will run applications that can
44
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
work also with low bandwidth, like web-browsing and measurement tools (ping, netperf, rina-tgen,rinaperf, ...).
Experimentation with routing policies tailored for mobility (that produce routing updates morefrequently), will explore good trade-offs for the maximum size of a DIF given a certain routingpolicy and a certain mobility rate (maximum size will be limited by the time to disseminate therouting updates when a handover happens). The influence of having a routing policy in every DIFon the routing table sizes will also be investigated.
Figure 14: Baseline LTE experiment
The first experiment will be a baseline measurement of the LTE user and data plane to be usedfor reference values. OpenAIRInterface (OAI) [16] is the software of choice as it is open source,easily available and has an active development team behind it. This experiment will also serve as anintegration test validating the OpenAIRInterface deployment. The deployment is shown in Figure14. It contains the following elements:
• The User Equipment (UE). The UE will run the OAI soft UE client, two end-user applications(VLC client and netperf).
• eNodeB’s (2). These two servers will run the OAI soft eNodeB module and the UE willperiodically move between the two base stations (simulated through disconnects over thephysical link).The two base stations are not directly connected since the X2 interface is notimplemented yet by OAI. Should such an interface become available during the project, thisexperiment might be rerun to include the results.
• the S-GW and P-GW. This will use the OAI soft EPC implementation.
• the MME, also from the OAI soft EPC implementation.
Each element will be installed on a bare metal server to provide accurate traffic measurementsin abscense of interference of schedulers associated with running elements in virtual machines. We
45
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
will capture the entire experiment’s traffic at all interfaces and analyse the traces to measure KPIsrelated to handovers and signaling overhead. A small sister experiment with the same topologywill be conducted on the wireless testbed to validate the results on a real radio link. The reasonfor this is that the wireless testbed is not always available (due to physical interference betweenexperiments, the testbed is usually entirely reserved for a single experiment at a time).
Figure 15: End-to-end LTE experiment
After this experiment, the deployment will be mirrored and run in both an LTE and RINAdeployment. The combined network is shown in Figure 15. Note that the eNodeB’s are directlyconnected, because this will have benefits for the RINA deployment. The eNodeB Clouds (eNBCs)will run on the same node and user data can be moved between these two nodes when the usermoves. Usage statistics will probably be monitord at the shim IPCPs on the eNodeBs. The MMEnodes will not be needed in the RINA setup.
The remainder of experiment2 will focus on scaling up the RINA scenario to 4 connected accessnetworks (each having 1 or 2 EPCs) with some 30 eNodeBCs between them, serving a combinedtotal of 1000 UE’s. The single metropop will be replaced by a 3 meshed MANs of up to 8 nodeseach interconnected by a core network of up to 5 nodes interconnected by (preferably) 10G links.The total final experiment will therefore use the following estimaed number of nodes in the testbed:5 core nodes, 20 metro nodes, 30 eNodeBC nodes and some 10 UE nodes simulating 1000 UE’s.
4.4 Metrics and KPIs
The objectives of this experiment are to quantify the following
• Handover interruption time: Handover interruptions can be derived from capturing data atthe application (for instance netperf or rina-tgen), and the physical interfaces of the deviceunder test. We will use tcpdump/pcap to capture the traffic traces.
• Packet loss during handovers: Packet loss can be derived in a similar way as the handoverinterruption time.
• PCI overhead: Protocol analysers such as Wireshark and tcpdump will allow us to separatecontrol packets in the IP over LTE scenario. In the RINA scenario, we will log statistics fromthe IPC processes themselves for analysis.
46
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• Routing scalability: Routing tables can be dumped to storage at certain intervals and all thechanges can be logged. These functionalities are already supported by the available RINAimplementations.
• Messaging overhead: We will use capture data from the physical interface to investigate themessaging overhead that a handover causes. Again we can use tcpdump/pcap to capture andanalyse the traffic traces.
• Average bitrate: Applications such as netperf or rina-tgen can be used to measure the averagebit rate to complete a file transfer.
These KPIs are summarised in Table 13.
KPI Metric Current state of the art ARCFIRE Objective
Handover interruption time ms 10-20 ms [17] ∼ 20 ms measured in the testbed
Packet loss during handovers number of SDUs 0 0
Routing scalability RIB entries IPv4 / IPv6 RIB/FIB Guarantee linear or sub linear scaling
Messaging overhead during handovers pps, b/s LTE CP traffic reduce control traffic
PCI overhead bits ROHC comparable to ROHC
Average bit rate to complete a filetransfer
Mb/s technology-dependent
Table 13: KPIs for experiment 2
4.5 Software
The software components that will be required for experiment 2 are listed in Table 14 for itsexecution. These include three available RINA implementations, the ARCFIRE measurement andexperimentation framework (Rumba, documented in [15]), various testbed-specific tools and allthe software necessary for the baseline LTE-based experiment. The possible use of different RINAimplementations is expected to carry an added value to the experiments, as different design decisionmay result in different performance and/or scalability properties.
4.6 Testbeds
The experiment will use the testbeds in Table 15 from the selection made in D4.2 [9]. Thesetestbeds have been chosen taking into account previous know-how of T4.3 partners, and to supportthe number of nodes required during the various phases of the experiment (i.e. ranging from ∼30to ∼1000 nodes). Moreover, running the experiment on both a VM-based testbed (GENI) and abaremetal-based testbed (Virtual wall) has been considered an important factor for the evaluation of
47
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software Description License
IRATI [5] Open source RINA Implementation for OS/Linux, inherited by FP7-IRATI andFP7-PRISTINE
GPLv2, LGPL
rlite [18] A lightweight Free and Open Source RINA implementation for GNU/Linuxoperating systems, focusing on stability, ease of use, and performance.
GPLv2, LGPLv2.1
Ouroboros POSIX compliant RINA implementation GPLv2, LPGLv2.1
jFed [19] Java-based framework for testbed federation MIT
Rumba [8] jFed compatible experimentation framework LGPLv2.1
OpenAIRInterface [16] Free Open Source Software for EPC, access network and user equipment of3GPP cellular networks
GPLv3
Quagga [20] Free Open Source routing platform GPLv2 (or later)
Table 14: Software for experiment 2
RINA software stability, as the sensitive difference in I/O timings 1 can result into a larger softwaretesting coverage.
Testbed Purpose
iLab.t Virtual wall Host the first experiments up to large scale testing. Measurements of bandwidth, delay, jitter andburst characteristics on dedicated links and bare metal hardware
w-iLab.t Validation of the LTE setup and measurements of eNodeB-UE bandwidth usage
GENI / planetlab Europe Emulation of a continent-wide core network, evaluation of scalability (routing table sizes) on VMs
Table 15: Testbeds for experiment 2
4.7 Planning
Table 16 details the expected milestones for experiment 2 throughout the execution of ARCFIRE.The first period (M16-M19) will be dedicated to the setup and KPI measurement for an LTE basednetwork, in order to collect statistics that can be used for later comparison with RINA experiments.From M19 on, experiments will make use of RINA, starting will small-scale setups (∼ 30 nodes) ina single testbed and scaling it up to ∼ 1000 nodes and using multiple testbeds.
The risk factors for experiment 2 are mostly related to the maturity of the experimentationsoftware. OpenAIRInterface and the used RINA prototypes do not (and cannot) have the samedegree of maturity as well-established IP-based and LTE software. This relative lack of maturitycan reveal itself in two ways. Moving an implementation from its development environment to thetestbeds may reveal corner cases (or in general code paths never taken before) that are related tothe specific hardware it is running on. Machines with different number of processors, processor
1I/O operations involving Virtual Machine can be an order of magnitude slower in terms of throughput and/or latencywhen compared to I/O on baremetal applications
48
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
Deploy OpenAIRInterface in iLab.t M16 Installed and verified the setup of an LTE-Advanced EPCand E-UTRAN with a single UE, 2 eNodeB’s, one MME,one S-GW, one P-GW and a core node hosting a singleservice
Connect two EPC deployments M17 The above deployment will be mirrored and connectedover a single core node
Document baseline measurements M19 This will give baseline measurements for the LTE sce-nario for the identified KPIs
Deploy RINA software in LTE scenario and get baselinemeasurements
M19 This will conclude the first small scale RINA experi-ment.
Scale up RINA deployment to mid-scale experiment M22 4 connected access networks, ∼ 20 UE’s, ∼ 10 eN-odeB(C)s, 2 services
Document measurements from the mid-scale experi-ments
M23 This will conclude the mid-scale experiments
Scale up RINA deployment to large-scale experiment M24 4 connected access networks, ∼ 1000 UE’s, ∼ 30 eN-odeBC, ∼ 10 services
Design multi-testbed deployment M25 This will depend on the results from the previous experi-ments
Execute multi-testbed deployment M27 This will depend on the results from the previous experi-ments
Wrap up experimentation M29 Detailed analysis of the results and preparation for publi-cation of the final results.
Table 16: Planning of experiment 2
49
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
microarchitecture, memory speed, caches size and NIC models can reveal timing issues and dataraces that were previously less visible. Scaling up the implementation will increase the rate atwhich any deficiency (memory leak, dereferencing a pointer just outside an allocated range) revealsitself. In addition to that, the likelihood of software failures increases (or at least does not decrease)as the duration of experiment increases.
As a quantitative indication of the largest and longest RINA experiments conducted to dateon the available implementations, IRATI has been scaled up on networks of Virtual Machinescontaining about 40 nodes; rlite has been deployed on networks of VMs with about 90 nodes.Moreover, these test networks have been operational only for some hours, while ARCFIRE targetsat least a week of uninterrupted operation.
These risks are mitigated in two ways:
• T3.4 (which runs until the end of the project) is dedicated to fixing bugs, memory leaks andother problems found during the various planned phases of the experiment.
• Three independent RINA implementations are available to be used for experiment 2, sothat fallback options are available if major issues should arise with either one of theseimplementations.
50
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
5 Experiment 3: End to end service provisioning across multiple net-work providers
5.1 Objectives
The main objective of this experiment is to assess the the RINA IPC API in terms of end-to-endservice provisioning complexity as compared to state-of-the-art network technologies currentlyin use in converged operator networks. This experiment will focus on the operation of severalservice DIFs supported by the core network of one or more service providers; these service DIFsprovide QoS guarantees to the applications using them, and request suitable QoS assurances ontheir supporting layers, the backbone DIF. Also, in order to support the guaranteed QoS levels, eachDIF will require specific sets of policies and parameters. Different scenarios will be considered,each one targeting different aspects of the experiment, and thus requiring different sets of policies,parameters, etc. These scenarios and the features to be taken into account are based on the analysiscarried out in deliverable D2.2 [2], sections 3.7 and 5.3. As discussed there, the focus will be onthe allocation of flows between applications or services by using their names, defining a contractfor an acceptable level of service for each flow in terms of technology-agnostic parameters; thismechanism will be shown to be of use between DIFs, allowing the communication of requirementsto the supporting DIFs and freeing the layers from the need to disclose internal technological detailsto the other layers. Allocation of flows between multiple operator networks with end-to-end qualityof service guarantees becomes possible and will be explored.
The second goal of Experiment 3 is to evaluate network renumbering. Multiple networkconfigurations will be analysed in which renumbering takes place in one or multiple DIFs at once.Dynamic renumbering has applications for mobility (moving hosts get new addresses that reflecttheir current location in the different DIFs they are members of) and also for modifying a DIFaddressing plan in case the network operator wished to do so.
Last but not least, Experiment 3 will analyse how RINA applies to an Open Mobile EdgeComputing (OMEC) use case. In OMEC use cases some applications may be hosted in the serviceprovider data centres distributed through the mobile access network. These applications are onlyavailable to certain uses via secure slices. This scenario will focus on the analysis of the DIFallocator to locate and application processes and create new DIFs, secure access to DIFs and RINAsupport for distributed mobility management to realise OMEC use cases.
In the following sections theses scenarios are described in detail.
5.2 Quality of Service levels guaranteed by service provider network
5.2.1 Objectives
Figure 16 depicts the core network of a service provider supporting multipurpose service DIFs. Inorder to design the best suited policies for this scenario, the distance between nodes and traffic
51
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
characteristics are the basic parameters. Two network sizes will be considered, a smaller one with11 backbone nodes and a larger one, with up to 40 nodes. For the latter, two service providerswill be considered, with the abstract layering structure shown in figure 17. The distance betweenthe nodes will be considered as added delay between the point-to-point links. Regarding thetraffic characteristics, each DIF will have to transport different types of traffic, with different QoSrequirements.
Figure 16: Guaranteed QoS levels experiment: abstract layering structure, showing the differentDIFs to be considered
The QoS guarantees would be achieved through a set of policies for the RMT and the EFCPimplementing the QTAMux, as described in D2.2 [2]. It provides quality differentiation andscheduling for different traffic classes.
Figure 17: Guaranteed QoS levels experiment: abstract layering structure, showing the differentDIFs to be considered. Provider edge routers are shown
The DIFs shown in figure 16 will have to deal with different traffic profiles, and thus the policies
52
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
for the RMT and EFCP would need to be adapted:
• Backbone core DIF will transport highly aggregated traffic. Thus, for this scenario only oneQoS level would be needed.
• The service provider DIF on top will transport several types of traffic. In this DIF is where theQTAMux enters the picture, providing the scheduling and delay/loss multiplexing requiredto satisfy the level of service requested by applications.
• Several VPN DIFs for different customers will be supported by the service provider DIF. Inthis case, the DIF will support the same QoS levels that the service provider DIF will offer,but no QTAMux will be required, as it will be the supporting DIF which will be doing that.
5.2.2 Metrics and KPIs
The KPIs for experiment 3 are given in Table 17.
KPI Metric Current state of the art ARCFIRE Objective
Delays Delays measured per QoS class Hard to guarantee differential delaybetween QoS classes at high loads[21]
Statistical bounds on delay per QoSclass, even when offered load isequal to 100%
Jitter Jitter measured per QoS class andflow
Hard to guarantee differential jitterbetween QoS classes at high loads[21]
Statistical bounds on jitter per QoSclass, even when offered load isequal to 100%
Losses Losses measured per QoS class Hard to guarantee differential packetloss between QoS classes at highloads [21]
Statistical bounds on packet loss perQoS class, even when offered load isequal to 100%
Table 17: KPIs for guaranteed QoS levels experiment
5.2.3 Software
This experiment will require the following software items for its execution, summarised in Table18.
5.2.4 Testbeds
The performance-sensitive nature of this experiment requires the use of bare metal hardwareresources if possible. Only a moderate number of resources is required to carry out the tests,without the need of supporting wireless interfaces. Hence, the Virtual Wall is the best testbed optionfrom the selection made in D4.2 [9]:
53
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software package Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
rina-echo-time [5] Measure jitter, delay and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
Table 18: Software required for guaranteed QoS levels experiments
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out the experiments described in this scenario
5.2.5 Experiment scenario
Two scenarios have been designed in order to gather data for the effect of network size in the KPIsdescribed above. Figure 18 shows the backbone connectivity of the small scale setup. For the largersetup, two similar backbone connectivities will be used for both network providers.
Figure 18: Guaranteed QoS levels experiment: small DIF experiment, backbone DIF connectivity
The most relevant characteristics of the policies for the backbone DIFs are summarised in thefollowing paragraphs.
54
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• Data transfer: No retransmission control policies will be considered.
• Support for multiple QoS classes: In these DIFs only one QoS class will be offered.
• Multiplexing and scheduling policies: Since the N-1 DIFs will be point to point DIFs, andonly one QoS class will be offered, a simple FIFO scheduling schema will be used.
The characteristics of the policies for the service provider DIFs are summarised in the followingparagraphs.
• Data transfer: No retransmission control policies will be considered. Sliding window flowcontrol, with ECN based congestion control will be used.
• Support for multiple QoS classes: In these DIFs, traffic from different applications or DIFs,with different QoS requirements will make necessary to offer several QoS classes. For thescenarios considered here, it will be necessary to offer the same classes as the layer on top,the VPN DIF or app-specific DIF or public internet DIF.
• Multiplexing and scheduling policies: For these DIFs it will be necessary to use theQTAMux policies, given the multiple QoS classes to be supported.
Finally, for the the VPN DIFs or app-specific DIFs or public internet DIFs the following policieswill be considered:
• Data transfer: In this DIF, taking into account the possibility of packet losses due to theQTAMux scheduling in the DIF below, retransmission control policies will be made availablefor the applications that require it. Since burstiness is to be expected, sliding window flowcontrol will be used, associated with ECN based congestion control.
• Support for multiple QoS classes: In this DIF, traffic from different applications withdifferent QoS requirements, will make necessary to offer several QoS classes.
• Multiplexing and scheduling policies: For this DIF no scheduling policies will be necessary.The DIF below will take care of the scheduling.
Table 19 shows the different types of traffic that will be considered in this scenario. Thesetraffic profiles will be translated into configuration parameters for the QTAMux.
Thus, three applications would be used to gather the results: real-time voice, video on demand,file transfer; each one is a modified version of the rina-echo-time application.
Figure 19 shows the layering for a scenario with two network service providers. The VPN DIFwill support the traffic profiles defined above matching them with its QoS classes. The Edge ServiceRouters aggregate traffic from different CPEs; therefore, each one will support several flows foreach type of application. These flows will be aggregated and transported by the backbone DIF.
55
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Urgency Type of application Flows allocated (%) Cherish level
Urgent traffic Real-time voice 20% Medium
Medium urgency traffic Video on demand, web brows-ing
40% High
Low urgency traffic File transfer 40% Low
Table 19: KPIs for guaranteed QoS levels experiments
Figure 19: Guaranteed QoS levels experiment: layering of the scenario
56
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
MS 1 M16 All software for experiments works on demonstrator, initial results using small-scale and large-scaledeployments
MS 2 M18 Experiments carried out at VWall testbed using small-scale deployment
MS 3 M22 Experiments carried out at VWall testbed using large-scale deployment for one network service provi-der
MS 4 M28 Experiments carried out at VWall testbed using large-scale deployment for two network serviceproviders
Table 20: Milestones for guaranteed QoS levels experiment
To carry out the experiments, many flows for each type of application will be allocated betweenall possible node pairs (in the case of the small scenario), and between nodes that are not tooclose in the case of the larger one. The delays, jitter, and packet losses will be measured by theapplications.
5.2.6 Planning
We foresee the milestones reported in Table 20 in the development of the guaranteed QoS levelsexperiment.
5.3 Dynamic and seamless DIF renumbering
5.3.1 Objectives
One of the requirements for achieving seamless mobility and at the same time keep routing efficientand forwarding tables small in DIFs is the ability to renumber IPC Processes as they move throughthe network. An effective and scalable addressing strategy assigns addresses to IPCPs that arelocation dependent (that is, the address reflects the position of the IPCP with respect to an abstractionof the DIF connectivity graph). As the IPCP moves through the DIF, it will reach a point whereits current address no longer reflects his position accurately (hence it is no longer aggregatable).The IPCP needs to get a new address assigned in a dynamic fashion, following a procedure thatguarantees that the flows supported by the IPCP are not impacted.
The procedure to change the address of a network entity is usually known as renumbering, andis not a seamless one in IP networks: IP addresses need to be assigned to interfaces on switches androuters, routing information must be propagated, ingress and egress filters must be updated - aswell as firewalls and access control lists, hosts must get new addresses and DNS entries have to beupdated.
An overview of the problems associated to renumbering of IP networks is provided in [22].Since TCP and UDP connections are tightly bound to a pair of IP addresses, changing any of themwill destroy the flow. Since DNS is an external directory - not part of the network layers - the
57
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
renumbering process usually leads to stale DNS entries pointing to deprecated addresses. Evenworse, applications may operate through the direct use of IP addresses, which will require anupdate in the application code, its configuration or both. Router renumbering usually requires anexhaustive manual and error-prone procedure for updating control plane Access Control Lists orfirewall rules. Moreover, static IP addresses are usually embedded in numerous configuration filesand network management databases [23].
Dynamic renumbering in RINA networks is possible because none of the previous issues canhappen. Flows provided by RINA DIFs are between application process names; since IPC Processesnever expose addresses outside of the DIF renumbering events are not externally visible. All DIFshave internal directories that match names of the applications registered to that DIF to the addressesof the IPC Processes they are currently attached to; therefore renumbering requires the update ofthis mapping, which is just a normal operating procedure in the DIF (no special function required).Access control rules are also setup according to application names, and therefore need not to beupdated when IPCPs change addresses. Last but not least, Network Manager applications interactwith Management Agents in the managed systems via its application name, and hence are notexposed to IPCP address changes.
The main goal of this experiment is to understand the behaviour of dynamic renumbering onRINA networks, analysing what are the limitations and tradeoffs to make renumbering events in aDIF invisible to the applications using it. This experiment will explore dynamic renumbering inseveral configurations of DIFs extracted from D2.2 [2], using this technique in two main use cases:
• Hosts moving through a provider’s access network. Addresses in the IPCPs of the mobilehost will need to be updated as it changes its point of attachment. In this scenario onlya subset of the addresses of IPCPs in one or more DIFs need to be updated, but that mayhappen rather frequently depending on the speed of the mobile hosts.
• A provider updates the addressing plan of one or more DIF(s) in its network. This mayhappen because the current addressing plan no longer scales, or because the provider wantsto change to an addressing plan that is more efficient given the network connectivity graph.In this scenario all the addresses of IPCPs in one or more DIFs need to be updated, but veryinfrequently.
5.3.2 Metrics and KPIs
The objectives of the experiment are to quantify i) the degradation in the level of service perceivedby applications using flows when renumbering takes place in a DIF and ii) the overhead andperformance of the renumbering procedure. As later explained in section 4.3.5, experiments willtake into account realistic scenarios but also more extreme scenarios to understand the limitationsof dynamic renumbering (e.g. in which a network is continuously being renumbered). That is whythe seamless renumbering KPI targets in table 21 refer to the realistic use cases.
58
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
KPI Metric Current state of the art ARCFIRE Objective
Seamless renumbering: Latency Increased latency while thenetwork is being renum-bered
Application flows breakwhen renumbering
Less than 5% in realisticuse cases
Seamless renumbering: Goodput Decreased goodput whilethe network is beingrenumbered
Application flows breakwhen renumbering
Less than 5% in realisticuse cases
Seamless renumbering: packet loss Packet loss due to renum-bering events
Application flows breakwhen renumbering
Zero in realistic use cases
Renumbering overhead Average extra entries inIPCP forwarding tablesdue to renumbering events,as a function of renum-bering period and DIFsize
Renumbering cannotbe fully automated andseamless
Understanding the trade-offs between renumberingperiod, DIF size and for-warding table size
Renumbering speed Time to complete a renum-bering event (until oldaddress is deprecated)
Renumbering cannotbe fully automated andseamless
Understanding the perfor-mance of the renumberingprocedure, and how it mayimpact seamless renumber-ing metrics
Table 21: KPIs for renumbering experiments
Software package Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGPL
Table 22: Software required for renumbering experiments
5.3.3 Software
This experiment will require several software items for its execution, summarised in Table 22.
• IRATI RINA implementation. It will be used to implement multiple RINA-enabled systemsand run a variety of DIFs according to the experiment configuration depicted in section 4.3.5.Policies already present in the IRATI repository will be used to carry out standard DIFprocedures. A few new namespace management policies will be developed to trigger therenumbering events according to different strategies (periodically, whole DIF at once, onlymobile hosts).
• rina-echo-time. The rina-echo-time application will be used to measure the latency andpacket loss perceived by a flow user while renumbering takes place.
59
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• rina-tgen. The rina-tgen application will be used to measure the goodput perceived by a flowuser while renumbering takes place.
• Demonstrator. The demonstrator will be used to test the correct behaviour of the softwarebefore deploying it on FED4FIRE+ testbeds. The demonstrator allows the experimenter tocarry out multiple experiment runs in a controlled environment with enough scale (up to 100RINA-enabled Virtual Machines).
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
5.3.4 Testbeds
We do not foresee the use of wireless resources to carry out the renumbering experiment, thereforethe Virtual Wall will be its main testbed of reference as reported in D4.2 [9]. The Virtual Wallcan provide access to bare metal hardware to run the RINA implementation and the applicationsrequired for the experiment, providing access to an adequate number of resources.
Initially the jFed experimenter Graphical User Interface (GUI) will be used to reserve and setupthe resources in the FED4FIRE+ testbeds. Once it becomes available, the experimentation andmeasurement framework under development in WP3 will provide a more automated front-end forjFed as well as other IRATI deployment and configuration tools such as the Demonstrator.
In summary, this experiment may use the testbeds in Table 5.3.4.
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out renumbering experiments
Table 23: Testbeds for renumbering experiments
5.3.5 Experiment scenario
Renumbering experiments will be based on two scenarios: single DIF scenarios, designed totest how renumbering behaves within a single DIF; and multi-DIF scenarios, designed to testrenumbering across a whole network (set of DIFs). We foresee two versions of the single DIFscenario: a small one and a large one. The first one, depicted in Figure 20, is modelled after theGEANT network backbone and has 32 IPC Processes. The second one, depicted in Figure 21, ismodelled after the AT&T network layer 3 backbone and has 80 nodes.
Multi-DIF experiments will leverage the ideal RINA converged service provider designsdescribed in D2.2 [2]. In particular we will setup small and large multi-DIF scenarios for the twouse cases considered in the experiment: mobility and change of addressing plan. Figure 22 shows
60
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
TALOslo STO
COP
DUB
LISMAD
Bern
Rome
SOF
VIE BUC
ATH
VIL
NIC
WAR
PRA
LUX
Riga
BER
LONAMS
BRU
PAR
LJU ZAG
BUD
ANK
VAL
MOSOslo
DUB
LIS
NIC
BRU
ANK
VAL
Riga
Figure 20: Renumbering experiment: small single DIF scenario, layering structure (up), and DIFconnectivity (down)
61
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
DAL
OKH
ORL
SJUMIA
FW
ALB
COL
PHO
DEN
COL
ATL
LA
LVG
SDG
SLC
AUS
SAN HOU
NOR
GAR
HON
SJO
SFR
SCR
DC
NYCDET
CAM
CHIOMH
KAN
SEA
ANC
POR
SLO
NORANH
RED
OAK
TMPFLA
PMJAK
SPO
MAD
MIL
DAV
DMO
NASHTUL
LRO
GLE
GRA
SPA
MIN
MEM
IND
LOU
BIR
CIN
CHA
RAL
PHIPIT
BAL
NEW
BOH
SYR
BRI
CHE
ALB
WOR
PORMAN
FAR
PRO
Figure 21: Renumbering experiment: large single DIF scenario, DIF connectivity
the DIF structure for the change of addressing plan scenario. The experiment will focus on theresidential customer DIF and below. Several tests will be carried on two versions of the scenario.The first one is illustrated by Figure 23, featuring 41 systems. The second one is depicted in Figure24, and uses 112 systems in a larger scale configuration. The scenario where renumbering supportsmobile hosts will be tested using a similar configuration, in which mobile hosts will be used insteadof CPE routers. Since we are only interested in analysing the change of address events, there isno need to have real mobility using wireless technologies in the experiment setup; mobility eventswill be simplified down to renumbering events occurring in a pattern equivalent to that of multiplemobile hosts roaming through the network.
A namespace management policy will be used to trigger different network renumbering situ-ations. With this policy every IPCP in the DIF periodically changes its address (walking a poolof addresses assigned reserved for each IPC Process). The period is a random variable uniformlydistributed between a minimum and a maximum values (configurable when specifying the policy).Playing with the configuration of the policy it is possible to experiment with DIFs where all itsmembers continuously renumber at different times, or DIFs where some of its members changeits address, or DIFs where all its members change addresses at the same time just once. Figure 25shows the different scenarios and configurations that will be taken into account for the renumberingexperiment.
62
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Access Router
PtP DIF
CPE
Edge Service Router
Metro Edge
Router
Metro Edge
Router
Metro BB DIF
Metro service DIF
PtP DIF PtP DIF
PtP DIF PtP DIF
Metro P router
PTP DIF
Residential customer service DIF
Host PtP DIF
Public Internet or App-specific or VPN DIF
Backbone Router
Backbone router
PtP DIF PtP DIF
Backbone DIF
Provider Edge
Router
Provider Edge
Router
PtP DIF
Customer network Service Prov. 2 Service Prov. 1 network
Access Aggregation Service Edge Core Internet Edge
Public Internet or App-specific or VPN DIF
Home DIF
Figure 22: Renumbering experiment: DIF structure for the multi-DIF scenario
Customer Premises Equipment
Access Router
MAN Access Router
MAN Core Router
Edge Services Router
Backbone router
Figure 23: Renumbering experiment: systems for small scale multi-DIF scenario
63
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Customer Premises Equipment
Access Router
MAN Access Router
MAN Core Router
Edge Services Router
Backbone router
Figure 24: Renumbering experiment: systems for large scale multi-DIF scenario
Continuous renumbering
Mobile host moves through the DIF
Renumbering experiments
Change of addressing plan
Single DIF
Multiple DIFs
Small scale
Large scale
Small scale
Large scale
Single DIF
Multiple DIFs
Small scale
Large scale
Small scale
Large scale
Small scale
Large scale Multiple DIFs
Figure 25: Categorisation of scenarios and configurations for the renumbering experiments
64
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
MS 1 M16 All software for experiments with continuous renumbering scenario works on demonstrator, initialresults using small-scale and large-scale deployments
MS 2 M18 Continuous renumbering experiments carried out at VWall testbed using small-scale deployment
MS 3 M22 Change of addressing plan and mobile hosts scenario experiments carried out at VWall testbed usingsmall-scale deployment (maps to ARCFIRE project milestone MS8)
MS 4 M26 Continuous renumbering experiments carried out at VWall testbed using large-scale deployment
MS 5 M28 Change of addressing plan and mobile hosts scenario experiments carried out at VWall testbed usinglarge-scale deployment (maps to ARCFIRE project milestone MS9)
Table 24: Milestones for renumbering experiments
5.3.6 Planning
We foresee the milestones reported in Table 24 in the development of the renumbering experiments.
5.4 Application discovery, mobility and layer security in support of OMEC
5.4.1 Objectives
OMEC can be defined as “An open cloud platform that uses some end-user clients and located atthe ‘mobile edge” to carry out a substantial amount of storage (rather than stored primarily in clouddata centres) and computation (including edge analytics, rather than relying on cloud data centres)in real time, communication (rather than routed over backbone networks), and control, policy andmanagement (rather than controlled primarily by network gateways such as those in the LTE core).”[24]
D2.2 [2] introduced the Open Mobile Edge Computing Scenario as one of the 5G scenarioswhere RINA could deliver significant value. This experiment analyses the buit-in capabilities inRINA that facilitate the realisation of OMEC use cases. In particular:
• Automatic location of applications regardless of the DIF(s) they are available through.In mobile edge computing scenarios storage, compute, and networking of edge servicesare provided at the edge of the mobile network. In a RINA network, compute and storageresources can be placed wherever needed. RINA can then discover distributed applications(edge services), locate processes and allocate flows to (between) them independent of theirnetwork location.
• Slicing: security and performance isolation. Network slices are isolated with guaranteedsecurity and performance, while sharing the same underlying infrastructure. All of that isoptimised for the delivery of a set of applications (or 5G verticals). RINA already has anative support for scope, slicing, and virtualisation as discussed in sub-section 3.8.1 of D2.2[2]. In essence, the concept of a DIF provides for securable layers (any number of them)
65
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
whose policies can be tailored to the needs of each tenant and/or application (e.g. virtualisednetwork function).
• Distributed mobility management. Distributed mobility management avoids centralisedmobility anchors providing efficient routing and traffic management (including handovers).Furthermore, it can be used to remove tunnels if possible. RINA already supports (natively)multi-homing and mobility, including multiple possible handover scenarios. Other RINAcore features (scoped routing, topological address and routing, and seamless renumbering)allow a RINA network to efficiently support any type of mobility and distributed mobilitymanagement. Experiment 2 will analyse this topic in depth, Experiment 3 only investigateshow distributed mobility management is used as an enabler of OMEC scenarios.
5.4.2 Metrics and KPIs
The objectives of this experiment are to evaluate the behaviour of several built-in RINA featuresfacilitating OMEC use cases, by verifying their correct operation and measuring the performanceof its current implementation. In particular we are interested in the DIF Allocator work to locatea destination application and configure DIFs to access it if needed; as well as in the impact ofhandovers in terms of performance (packet loss, delay, goodput) in a RINA over WiFi scenario(Table 25).
KPI Metric Current state of the art ARCFIRE Objective
DIF Allocator performance Extra delay incurred inflow allocation due toapplication discovery andjoining relevant DIF
This capability is notsupported by the currentInternet protocol suite
Understanding the perfor-mance of the DIF Allocatorunder load, identifyingtrade-offs in its design
Impact of handover on packet loss Increased packet loss dueto handover (WiFi AP toWiFi AP)
To be measured on testbed Equivalent or less than inthe IP case (understandthe tradeoffs of RINA overWiFi)
Impact of handover on delay Increased delay due tohandover (WiFi AP to WiFiAP)
To be measured on testbed Equivalent or less than inthe IP case (understandthe tradeoffs of RINA overWiFi)
Impact of handover on goodput Loss of application good-put (Mbps) due to handovereffects
To be measured on testbedfor TCP and UDP
Equivalent or less than inthe IP case (understandthe tradeoffs of RINA overWiFi)
Table 25: KPIs for OMEC experiments
5.4.3 Software
This experiment will require several software items for its execution, summarised in Table 26.
66
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Software package Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
DIF Allocator [5] Locates applications over a variety of DIFs and collaborates with NMS(s) tocreate new DIFs
TBD
rina-echo-time [5] Measure latency and packet loss GPL
rina-tgen [6] Traffic generator, measure goodput GEANT OS License
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Rumba [8] Tool to setup RINA implementation experiments on FED4FIRE+ testbeds LGP
Table 26: Software required for OMEC experiments
5.4.4 Testbeds
The w-iLab.t wireless testbed provides an ideal environment for the wireless segment of theexperimental scenario depicted in Figure 26. Such a facility provides access to a large number ofWiFi access points - which will be used to model the access routers in the experiment - as well asaround 20 hosts mounted on robots that can move through the facility - used to model the mobilehosts. The Virtual Wall will be used to provide resources for all the fixed routers (core routers, DCgateway, Top of Rack, ISP routers, provider border router) as well as the servers in the experiment.There is connectivity between the w-iLab.t and the Virtual Wall, therefore it is possible to carry outexperiments that use resources from both testbeds.
Table 5.4.4 summarises the testbeds that will be used by the OMEC experiment.
Testbed Purpose
w-iLab.t Hardware resources for the wireless access routers and the mobile hosts, connectivity to Virtual Wall
Virtual Wall Hardware resources for the fixed routers (core, border, ISP, DC) and servers
Table 27: Testbeds for OMEC experiments
5.4.5 Experiment scenario
The experiment scenario features a provider hosting applications belonging to two different en-terprises in its own datacentre. Several mobile hosts (UEs) belonging to employees of each oneof the two companies are serviced by the provider. UEs connect to applications that are eitheravailable through the public Internet or through the provider’s edge datacentre. UEs are completelyunaware of the location of such services, which are dynamically discovered by the RINA network.Employees can move and attach to different provider access routers without service disruption.Figure 26 shows the configuration of the physical systems in the experiment scenario. The providernetwork consists in 6 wireless access routers, interconnected between them via 2 core routers,
67
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
which in turn interconnect to the provider network border router. This border router is connected totwo ISPs (modelled with a single router since the details of the ISP network are not important forthis experiment), each one are attached to a server. A datacentre owned by the network provider isattached to one of the core routers.
UE 1
UE 2
AR 1
AR 2
AR 3
AR 4
AR 5
AR 6
CR 1
CR 2
GW 1
ISP 1
ISP 2
SRV 5
SRV 6
SRV 1
SRV 2
SRV 3
SRV 4
ToR 2
ToR 1
DC GW
Small DC
Service Provider net
Data Center Gateway
User Equipment
Provider Access Router
Core Router
Provider 1 Border Router
ISP Router
Server
Top of Rack Router
Figure 26: OMEC experiment: physical systems involved in the scenario
The DIF structure of the experiment for the communication between applications in UEs andapplications in public Internet facing DIFs is shown in Figure 27. The first hop between mobilehosts and the access network is serviced by shim DIFs over WiFi, which allow higher layer DIFsand local RINA management to interact with the WiFi protocols (triggering access point scanning,association and dissociation). On top of that the mobile network DIF provides distributed mobilitymanagement across the service provider network. Finally, a public Internet DIF allows applicationsin UEs to reach applications hosted at servers outside of the service provider network.
Figure 28 shows the DIF structure for the communication between applications in UEs andapplication hosted at the service provider private cloud. A DC Fabric DIF connects all the servers inthe service provider datacentre to the datacentre border router (labelled gateway in the Figure). TwoVPN DIFs float on top of the DC Fabric DIF, each one providing private access to the applicationsof a different company. These VPN DIFs span all the way to the UEs when those need to connectto applications hosted at the provider’s datacentre.
The experiment will analyse the following scenario:
1. Once a given mobile host has joined the Mobile Network DIF, the rina-echo-time application
68
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
UE Access 1 Core 1 Gateway
ISP1 Server6 Mobile network DIF
Internet DIF
DAF (rina-tgen or rina-echo-time)
Shim DIF WiFi Shim DIF Eth Shim DIF Eth
Shim DIF Eth Shim DIF Eth
UE A1
A4
A2 C1
GW
C2
Mobile Network DIF
I1
UE GW
S1
S2 I2
Internet DIF
A3
A5
A6
DC
Figure 27: OMEC experiment: DIF configurations: UE to server on the public Internet
69
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
UE Access 2 Core 1 DC Gateway
ToR 1 Server 1
Mobile network DIF
Enterprise 1 VPN DIF
DAF (Any demo app)
Shim DIF WiFi Shim DIF Eth Shim DIF Eth Shim DIF Eth Shim DIF Eth
S2
GW
S4 Enterprise 2 VPN DIF
DC Fabric DIF
S1
UE 1
GW
S3 Enterprise 1 VPN DIF
UE 2 S2
S1
S3
S4
GW
ToR1
ToR2
Figure 28: OMEC experiment: DIF configurations: UE to server on the provider’s cloud
70
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
in the UE will request a flow to a rina-echo-time server hosted in one of the servers outsideof the provider’s network.
2. Then the DIF Allocator will start the search for the DIF that brings to the rina-echo-timeserver, locating it and figuring out that the destination application is available through thepublic Internet DIF.
3. After that the DIF Allocator instance at the UE will create an IPC Process and instructs it tojoin the public Internet DIF.
4. Once it has joined the flow allocation request will be passed to the public Internet DIF, whowill allocate the flow.
5. A second flow to another application hosted in a public Internet facing server will be requested(now by the rina-tgen application). Time to flow setup in this case should be lower, since themobile host is already a member of the public Internet DIF.
6. While the two applications are executing, the UE moves and changes its attachment to theservice provider’s access routers.
7. Now the rina-echo-time client requests a flow to a rina-echo-time server hosted at theprovider’s private cloud.
8. The DIF Allocator will look for the destination application, and will realise it is availablethrough the Enterprise 1 VPN DIF.
9. Then the DIF Allocator instance at the UE creates an instance of an IPC Process and instructsit to join the Enterprise 1 VPN DIF (requires authentication), who later processes the flowrequest and allocates the flow.
The experiment will be carried out with a varying number of concurrent requests, starting froma single one (to verify the correct operation of all the procedures involved in the scenario). Wewill measure how the increased load in the Flow Allocator increases its response time, and howhandover performance is degraded. This data will measure the maturity and performance of theIRATI implementation - not of RINA per se, but this is also an important goal of IRATI.
5.4.6 Planning
We foresee the milestones reported in Table 28 in the development of the OMEC experiments.
71
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Milestone Month Description
MS 1 M18 Initial version of the experiment works with pre-allocated DIFs (no DIF Allocator) and 2 concurrentmobile hosts in local testbed
MS 2 M22 Experiment works with pre-allocated DIFs and 10 concurrent mobile hosts in w-iLab.t and VWalltestbeds (maps to ARCFIRE project milestone MS8)
MS 3 M26 Experiment works with DIF Allocator (dynamic DIF creation) and 2 concurrent mobile hosts in localtestbed
MS 5 M28 Experiment works with DIF Allocator (dynamic DIF creation) and 10 concurrent mobile hosts in w-iLab.t and VWall testbeds (maps to ARCFIRE project milestone MS9)
Table 28: Milestones for renumbering experiments
72
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
6 Experiment 4: Studying the effects of (Distributed) Denial of Ser-vice (D)DoS attacks inside and over RINA networks
6.1 Introduction and motivation
Distributed Denial of Service (DDoS) attacks make illicit use of a number of third-party computersto overwhelm one or more resources which are reachable through the Internet. DDoS attacks havematured and intensified during the last years [25], developing into an industry in which customerswanting to set up an attack can find DDoS toolkits by searching Google or, even easier, can pay afew tens of dollars per month to individuals providing DDoS attack services for hire. The growingnumber of cheap, insecure IoT devices connected to the Internet has contributed to scale up thesize of DDoS attacks, with record attacks disrupting personal websites (Brian Krebs on Security[26]) or Internet infrastructure companies providing DNS services (the Dyn attack [27]) havingbeen reported during the last months of 2016.
To launch a successful DDoS attack, first the attacker has to assemble an army of infectedcomputers that run the malicious software in charge of performing the attack, called a botnet. Oncethe botnet is ready, the attacker instructs it to attack one or more targets (websites, gaming servers,etc.) via command and control servers that control the operation of the botnet.
6.1.1 State of the art in botnet formation
There are multiple ways to infect a computer with a worm, trojan or other malicious software:spam, software download, insecure default settings, etc. As a representative example we will focusour analysis on the vulnerabilities exploited by the Mirai botnet, which participated in bot the Dynand Krebs website attacks. Mirai scans the Internet public address space looking for connecteddevices that have their Telnet and SSH ports open. It then tries a number of default passwords (setby the device vendors when they ship their products) from an internal dictionary, trying to gainaccess to the device. If successful, it downloads the botnet software and installs it into the infecteddevice (usually IoT devices such as security cameras, DVRs, etc., but also residential broadbandrouters). In particular Mirai exploits the following vulnerabilities:
• A global, public layer where all applications are reachable. The public Internet addressspace provides global reachability to all devices connected to it with a public IP address.Its address space is exposed to applications, therefore any application in the public Internetcan scan the whole Internet address space. Administrators that wish to restrict connectivityto/from applications on their sites have to configure a number of firewall rules so thatonly communication with authorised addresses is possible. Another option to restrict theconnectivity is to setup VPNs for different application groups (e.g. IP security cameras donot need to be reachable by the whole Internet), hence authentication would be required to beable to send traffic to the devices in such VPNs. However, setting up firewall rules or VPNs
73
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
is usually beyond the technical capability of most administrators in home or small companynetworks, which usually host the devices that are infected by the botnet.
• Use of well-known ports. The use of well-known ports instead of application namesfacilitates the large-scale scanning of Internet-connected devices, thus growing their chancesof being infected with the botnet malware. The risk can be mitigated by closing the remoteadministration ports (SSH, Webserver) that are usually open by default in this type of devices,or setting up firewall rules that only allow connections to these ports from within the localarea network. Again these techniques require some knowledge by the network administratoras well as his/her willingness to act.
• Use of insecure credentials. Since an attacker can reach the Internet-connected device withno authentication in the network layers, the only line of defense between him and the deviceare the credentials granting access to the device administration (in the application layer). Oneof the big problems with cheap IoT connected devices are that they usually come from thefactory with default credentials that are complicated to change: either they are embedded inthe firmware, or there are different credentials depending on the way of accessing the device(Webserver, SSH), or users just do not feel compelled to change the default password. Thissituation makes it trivial for a botnet to try to login several times using a dictionary-basedattack, until it finds the successful default password.
• Global address space • Addresses exposed to apps
• No authentication
• Well-known ports open • No firewall rules nor VPNs
• Insecure default administration credentials
Attacker Compromised Device
Public Internet
Figure 29: Vulnerabilities that enable malware to take control of Internet-connected IoT devices
6.1.2 State of the art of DDoS attacks
Once the botnet has infected enough computers it is ready to carry out the DDoS attack against oneor more targets. There are several types of DDoS attacks, all based on the premise of abusing theresources of the system under attack so that it can no longer operate on normal conditions (or evenhas to stop operating). Many of the attacks leverage IP address spoofing techniques, which allowthe sender to generate IP packets with fabricated source or destination IP addresses to make theattack more effective. The most popular types of DDoS attacks are:
74
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
UDP flood attacks. These attacks leverage the lack of flow control in the UDP protocol, whichmeans the receiver (system under attack) cannot limit the amount of information that the sender(attacker) is generating. Some of the attacks belonging to this category are:
• DNS reflection (or amplification) attacks. Attackers use publicly accessible open DNSservers to flood a target system with DNS response traffic. The primary technique consistsof an attacker sending a DNS name lookup request to an open DNS server with the sourceaddress spoofed to be the target address. When the DNS server sends the DNS recordresponse, it is sent instead to the target. Attackers will typically submit a request for asmuch zone information as possible to maximize the amplification effect, resulting in largeUDP packets sent to the target system. The root problem is that DNS does not authenticatethe requestor, therefore it cannot know that the source IP address was forged. Mitigationtechniques involve packet filtering at the ISP level to reject packets with source addressesnot reachable via the actual packet’s path [28]. Another mitigation technique is for the DNSserver to apply rate limiting on DNS responses, allowing a maximum number of responsesper second per IP address.
• NTP reflection attacks. This type of attacks are based on the same premises as DNSreflection attacks: use of an application protocol with optional authentication (thereforethe source IP address can be forged), that works on top of UDP (no flow control) and thatproduces a relatively large response to a small request (the size ratio of response vs. requestis called the amplification factor). What changes is the use of NTP (the Network TimeProtocol) instead of DNS. One of the NTP commands produces an answer that returns up to600 addresses of the latest servers NTP has interacted with; this can lead to amplificationfactors of up to x209 (the typical DNS amplification factor is x8). The easiest mitigationtechnique is to use NTP authentication.
GRE (Generic Routing Encapsulation) flood attacks. GRE mainly encapsulates data packetsand routes them through the tunnel to a destination network that de-encapsulates the payload packets.Sending many GRE packets with large amounts of encapsulated data may lead to extreme resourceconsumption when the victim tries to de-encapsulate them.
TCP SYN flood attacks. These attacks abuse the 3-way handshake by sending a TCP SYNpacket but never replying to the TCP-SYN-ACK with a TCP-ACK. This way they leave theconnection half-open in the server for some time (consuming resources) before it can be destroyed.The attacker sends another TCP-SYN packet before the connection is reset, therefore the totalnumber of “half-open” connections keeps growing until the target system can no longer acceptnew TCP connections. Mitigations to this attack include adjusting the TCP reset timers, allocatingmemory to the TCP connection only when the server receives the TCP ACK packet or using cookiesto check that the client is legitimate.
IP Fragmentation attacks. This type of attacks generate a number of manipulated segmentsof fragmented IP datagrams to cause problems in the reassembly process at the system under
75
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
attack (overlapping segments, segments that are too small, large number of incomplete packets,reassembled datagrams that are larger than the maximum size of an IP packet, etc.). These attackscan be mitigated by DDoS protection systems sitting at the network entry point that check forincoming packets that break IP fragmentation rules.
ICMP flood attacks. Similar to UDP flood attack, an ICMP flood overwhelms the targetresource with ICMP Echo Request (ping) packets, generally sending packets as fast as possiblewithout waiting for replies. This type of attack leverages the lack of flow control for ICMP packets.
6.1.3 State of the art of mitigation techniques of DDoS attacks
A comprehensive survey of the methods to defend against DDoS flooding attacks in the Internet isprovided in [29]. This survey classifies the different techniques to defend against DDoS floodingattacks depending on where they are deployed and enforced (close to the attacker, close to the targetor in intermediate networks) and when they are used: before, during or after the attack.
Source Network
Retail provider 1Transit provider Retail provider 2
Target Network
Target host
Compromised hosts
CPE CPEAccess router Access
router
Source-based defense mechanisms
Target-based defense mechanisms
Network-based defense mechanisms
Hybrid defense mechanisms
Figure 30: Classification of DDoS defense mechanisms depending on their deployment
Source-based techniques. Ingress/egress address filtering tries to limit the amount of trafficwith spoofed IP addresses leaving or entering the network (by checking that the IP address prefixis a valid one according to the network addressing policy). Traffic analysis samples inbound andoutbound network traffic, comparing it to models of normal traffic operation to detect potentialDDoS attacks. Reverse firewalls limit the outgoing network packets per destination, trying toprevent floods of packets to specific targets. In general source-based techniques have trouble indetecting DDoS attacks due to the very nature of the attack, since infected devices don’t generatevery high loads nor abnormal traffic patterns. There is also an economic problem regardingdeployment, since these mechanisms have a cost (memory/CPU utilisation) and do not protect thenetworks of the ones who deploy them (they do protect other networks).
Destination-based techniques. IP traceback mechanisms try to trace back the forged IPpackets to their true source address rather than the spoofed one, either using packet markingor link testing techniques. Management Information Base (MIB) analysis can be performed todetect abnormal patterns in the statistics of received ICMP, UDP and TCP packets. The MIB also
76
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
enables the modification of the router behaviour to react to the attack. Packet marking and filteringmechanisms aim to mark legitimate packets at each router on the path to the destination so thatthe target’s edge routers can filter the attack traffic. The marking can be based on informationregarding the history of known IP addresses, hop counts associated to legitimate IP addresses orpath identifiers embedded in IP packets. Packet dropping based on the level of congestion is basedon selectively dropping suspicious PDUs in routers once a certain threshold of congestion has beenreached. The problem with destination-based techniques is that they cannot detect and respond tothe attack until a significant number of the target resources are already being abused.
Network-based techniques. Route-based packet filtering extends ingress filtering to the coreInternet routers The other set of network-based techniques are based on detecting and filteringmalicious routers, based on analysing the traffic they forward and comparing it to a model of theexpected behaviour. These type of techniques require a significant number of memory and CPUresources, in addition to a large communication overhead between routers to coordinate; makingcurrent network-based approaches usually not effective nor efficient for fighting back against DDoSattacks.
Hybrid techniques. These techniques are deployed at multiple points, requiring coordinationbetween different components in order to be effective.
• Hybrid packet throttling techniques usually place packet detection close to the target andperform throttling close to the source of the attack. Aggregate-based congestion controland pushback compute maximum rates for flow aggregates and send pushback messages toupstream routers for the aggregates that are misbehaving. Upstream routers apply the sameapproach recursively. TRACK uses a port-marking module in routers that probabilisticallymarks packets with a local port-identifier. Systems under attack can use the informationin the packets to trace the attack back to the source, while upstream routers can filter themalicious packets.
• DEFensive Cooperative Overlay Mesh (DEFCOM) is a framework to enable the nodesparticipating in the DDoS defense to communicate. DEFCOM organises such nodes in apeer to peer network that follows an approximation of the underlying connectivity graph.DEFCOM classifier nodes differentiate legitimate traffic from attack traffic and are placedclose to the attack targets, while nodes that rate-limit malicious traffic are placed closer todestination.
• COSSACK leverages ingress/egress route filtering, attack signature matching and a multicastcommunication architecture to provide a target-identification and source-mitigation defensestrategy.
• Capability-based mechanisms let the destination explicitly authorise the traffic it desires toreceive. Destinations grant senders the capability of sending traffic via tokens that sendersmust embed in the packets they generate. Verification points along the path check if the
77
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
traffic is legitimate or not. These systems depend on securely granting capabilities, whichis usually addressed by authenticating the source (requiring routers to store cryptographicmaterial to validate legitimate packets).
• Active Internet Traffic Filtering (AITF) assumes the receiver accepts all the traffic andexplicitly denies the traffic that has been considered harmful. The receiver has to contact therouters of the ISPs that host misbehaving hosts so that their traffic is dropped.
Hybrid techniques are the most effective way to detect DDoS flooding attacks and mitigatingtheir effects, at the cost of having to deploy and coordinate DDoS defence mechanisms in multiplenetworks. Combining source address authentication (to prevent spoofing), capabilities and filteringcould provide the most effective and efficient solution. However, there will be a trade-off betweenperformance and accuracy in any DDoS defence solution.
6.2 Objectives
The previous sections introduced the state of the art of DDoS attacks that take place over theInternet, analysing some of the vulnerabilities that are exploited by the attackers. Ultimately,the ability to DDoS any shared resource depends on how many client actors can generate (somekind of) traffic toward it, and whether it is possible to source-control it or if it has to be sourcecontrolled at the destination. The high-level goal of this experiment is to investigate how the RINAenvironment is different from the Internet when it comes to DDoS attacks. Is it more resistant toDDoS attacks? Are there similar vulnerabilities to what we see in the Internet? The experimentwill also explore different configurations of RINA-based operator networks that may have differentlevels of effectiveness in preventing and mitigating DDoS attacks.
In this experiment we will focus on abusing the resources of properly-operating IPCPs. Clientscan cause the generation of a lot of management traffic by requesting flows (hitting directories andDIF allocators), generating data traffic via an established flow, or exhausting remote resources by(at a rate that can still be serviced) creating an increasing number of DIFs or flows within a DIF toa shared destination and never removing them. The number of attacking clients and the scale of theshared receiver(s) being attacked determines how serious each of these can be.
Our hypothesis is that the following properties of RINA can be leveraged to construct networksthat are less prone to the effects of DDoS attacks than the Internet is.
• Enrollment enforces explicit association control. In the internet, any pair of addressesis “already-connected” with respect to the ability to send packets to each other. The onlypairwise control is what can be imposed via routing redirection or firewalls after the fact,but neither stops the senders, it just protects the receiver. In a RINA network joining aDIF requires authentication. Before an application instance is able to send data to anotherapplication instance a flow needs to be allocated, allowing the DIF that allocates the flow toperform access control and police flow allocation requests.
78
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
• Use of application names. DIFs do not expose addresses to the applications using them. Nowell-known ports are used, each application has a location-independent application namethat identifies it.
• Congestion control at lower layers enables earlier detection and response. In the Internetcongestion control is only end-to-end at a per flow level. Hence, congestion caused by theaggregate traffic from multiple flows (like in a DDoS attack) can only be detected end-to-end.With RINA congestion control happens at each layer, with lower layers reacting to congestiondue to the aggregated traffic of multiple flows at higher layers [30]. Therefore in RINAnetworks it should be possible to detect abnormal volumes of traffic in lower layers closer tothe attackers, whereas in the Internet this can only be done close to the target of the attack.
The experiment will also try to provide answers on how more difficult is to discover vulnerableapplications in RINA networks compared to the public Internet. A bot in a RINA network hasno address space to scan, nor a list of well-known ports to probe: all it can try to do is to requestflow allocations to an application name. Therefore the namespace to “scan” is an applicationnamespace, which in most cases has a much greater scope than the public Internet address space.Insecure IoT devices such as the ones exploited in the DDoS attacks reported in the previoussections would register their “remote administration application” to one or more DIFs. Assumingthe same application protocols are used (Webserver and SSH), each IoT device would register twoApplication Entities (one for each protocol). The attacking bot would have to guess the applicationnames, and avoid blocking of the Allocate Request by the DIF Allocator (if allocating the flowfrom another DIF) or the Flow Allocator at the gateway/CPE of the user network.
6.3 Experiment scenario
Experiment 4 will investigate two general scenarios. In both scenarios we will consider 4 operatorswhich provide a variety of IPC services to end customers, all interconnected via a wholesale transitprovider. The first scenario, depicted in Figure 31, assumes a single top level DIF spanning allcustomers and operators where all applications reside. This configuration is equivalent to the publicInternet scenario, in which most applications are in a single top-layer network. Different configu-rations of attackers and targets will be investigated (geographically concentrated vs. distributedattackers, geographically concentrated vs. distributed targets). The second scenario, depicted inFigure 32, assumes multiple top-level DIFs supporting a variety of applications: security monitor-ing, customer VPNs, e-mail and web-surfing, video streaming, etc. In this scenario applications aresegmented across various DIFs, therefore hosts infected with bots will belong to different DIFs.Bots need to join different DIFs to mount an attack with the same scale as in the first scenario.
Figure 33 shows the DIF models for Experiment 4. Customer DIFs will have hosts directlyconnected to customer border routers (labelled CPEs in the Figure). Service provider networks aremodelled using a simplified version of the converged operator network design provided in D2.2 [2].
79
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Transit provider
Provider 3
Provider 1
Provider 2
Provider 4
Customer
Customer
Customer Customer
Customer
Customer Customer Customer
Customer
Customer Customer Customer
Customer
Customer Customer
Customer
Customer Customer
Public Internet DIF
Target AP Bot AP DIF Legend
Public Internet DIF
Customer DIF
CPE
Compromised Host Transit
BorderProvider Border
Transit Border
Provider Border
Internal DIFs
. . . PtP DIF PtP DIF PtP DIF PtP DIF PtP DIF PtP DIF
Access Router
DIF
Internal DIFs Internal DIFs
Access Router
PtP DIF
CPE
Customer DIF
Target Host
Figure 31: Experiment 4 scenario 1: single top-level DIF (public Internet) configuration
Transit provider
Provider 3
Provider 1
Provider 2
Provider 4
Customer
Customer
Customer Customer
Customer
Customer Customer Customer
Customer
Customer Customer Customer
Customer
Customer Customer
Customer
Customer Customer
Security monitoring DIF
Video streaming DIF
Web DIF
Customer VPN DIF
Target AP Bot AP DIF Legend
Security Monitoring DIF
Customer DIF
CPE
Compromised Host Transit
BorderProvider Border
Transit Border
Provider Border
Internal DIFs
. . . PtP DIF PtP DIF PtP DIF PtP DIF PtP DIF PtP DIF
Access Router
DIF
Internal DIFs Internal DIFs
Access Router
PtP DIF
CPE
Customer DIF
Target Host
Customer VPN DIF
Figure 32: Experiment 4 scenario 2: multiple top-level DIFs configuration
80
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
The model consists of one or more DIFs used by applications (the public Internet one or multipleapplication specific DIFs) running on top of an internal DIF.
CPE
. . .
Host
Host
Host
Model for the customer DIFs
. . .
. . .
. . .
CPE
CPE
CPE
CPE
CPE
CPE
PAR
PBR
PBR
Model for the Provider Internal DIF
Model for the Transit Provider Internal DIF
TBR
TBR TBR
TBR
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . . . . .
. . . . . .
. . .
. . .
Model for the Public Internet DIF
Host
Host
Host
Host
Host
Host
Host
Host
CPE
CPE
PBR
PBR
PBR PBR
PBR PBR
PBR
PBR
TBR
TBR
TBR
TBR
PAR
CPE
CPE
CPE = Customer Premises Equipment
PAR = Provider Access Router
PBR = Provider Border Router
TBR = Transit Provider Border Router
N-1 DIF
N-1 flow IPC Process
Figure 33: Models for the different DIFs present in the DDoS experiments
This configuration is adequate for the DDoS experiment, since it will allow us to look at thebehaviour of aggregate flows in the internal DIF of the provider while DDoS attacks take place.The internal DIF aggregates traffic of customer CPEs via access routers into the provider borderrouters, where the higher-level DIFs transporting application traffic are available. The transit DIF ismodelled as four IPC Processes in a full-mesh configuration. Finally, the public Internet DIF floatson top of the other DIFs, providing connectivity to applications running in hosts via CPEs, ProviderBorder Routers and Transit provider border routers. The model for the application-specific DIFs inscenario 2 is not shown because it is similar to the public Internet DIF but with each individualapplication specific DIF having a smaller scope.
The experiments in WP4 will be carried out over two configurations, labelled small-scale andlarge-scale. The systems involved in the small-scale configuration is depicted in Figure 34. 41systems are organised in two service providers, each one servicing 4 customers with 3 hosts. Thetwo providers are interconnected by a transit provider that has 3 border routers. A diagram of
81
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Host
Host
Host
CPE
Host
Host
Host
CPE
AR
Host
Host
Host
CPE
Host
Host
Host
CPE
AR
PBR TBR TBR
TBR
PBR
Host
Host
Host
Host
Host
Host
Host
Host
Host
Host
Host
Host
CPE
CPE
CPE
CPE
AR
AR
Cus
tom
er 1
Provider 1
Transit Provider
Provider 2
Cus
tom
er 2
C
usto
mer
3
Cus
tom
er 4
Cus
tom
er 5
C
usto
mer
6
Cus
tom
er 7
C
usto
mer
8
CPE = Customer Premises Equipment
PAR = Provider Access Router
PBR = Provider Border Router
TBR = Transit Provider Border Router
Figure 34: Systems in the small scale experimental scenario
82
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
the connectivity between the systems in the large-scale configuration is shown in Figure 35. Thescenario features 4 network providers servicing 4 or 6 customers with 3 hosts each one, making atotal count of 100 systems.
Customer Premises Equipment
Host
Provider Access Router
Provider Border Router
Transit Provider Border Router
Transit Provider
Provider 3
Provider 2
Provider 1
Provider 4
Cus
t 1
Cus
t 2
Cus
t 3
Cus
t 4
Cus
t 5
Cus
t 6
Cus
t 7
Cus
t 8
Cus
t 9
Cus
t 10
Cus
t 11
Cus
t 12
Cust 13 Cust 14 Cust 15 Cust 16
Cust 17 Cust 18 Cust 19 Cust 20
Figure 35: Systems in the large scale experimental scenario
Each scenario will be tested in two configurations: in the first one all networks will not berunning any policy specifically designed to identify and mitigate DDoS attacks. Only standard RINApolicies performing scheduling, flow control and congestion control will be used in the differentnetworks. By analysing the result of the experiment runs, we will identify the vulnerabilities thatcan be mitigated by specific policies (such as rate-limiting of flow allocation requests, or limitingthe remote applications that can discover and connect to local applications) and what informationcan be used to identify the sources of the DDoS attack. Such policies will be added to the secondconfiguration, thus allowing us to compare the complexity and scalability of the solutions deployedwith the gains obtained with regards to the KPIs described in Section 6.4. The experiment willfocus on two configurations for the target application. The first one, the standalone target, willtest a single instance of the target application, in a deployment equivalent to a DDoS attack to asimple personal website (such as the attack to the Krebs blog [26]). The second target applicationconfiguration, called distributed target, features a deployment equivalent to the attack to the DynDNS infrastructure [27]. In such configuration there are application instances that can do the task of
83
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
he Application (worker APs) and many request handling AP instances spread around the network.Allocates to the distributed application are forwarded to the “nearest” DAP member. If the zombiedistribution showed some locality, such as mostly from North America, etc. this would allowrequests to be distributed to request handling APs so that some of them would be under attack andothers would not be. The request handling APs could forward valid requests, i.e. those that getthrough CACEP authentication, to one or more of Worker APs that do the real application work ormay do it themselves.
8
Scenario 1 (Public Internet DIF)
Standard policies
Standalone target Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
Distributed target
Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
DDoS-specific policies
Standalone target Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
Distributed target
Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
Scenario 2 (App-specific DIFs)
Standard policies
Standalone target Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
Distributed target
Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
DDoS-specific policies
Standalone target Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
Distributed target
Botnet in all operators
Botnet in 2 operators
Botnet in 1 operator
DDoS Experiments
DIF Configurations Type of policies Type of target Distribution of bots
Figure 36: Categorisation of DDoS experimental scenarios
Three botnet configurations will be tested for each type of target application deployment: onein which all the bots are located in a single provider, a second one in which the bots are spreadacross two providers and a final one in which the bots are evenly distributed across all the providerspresent in the scenario. Figure 36 summarises all the scenarios that will be tested under Experiment4. Different runs with a growing number of bots will be performed to measure how the attacks scale
84
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
as the number of bots increases.Several countermeasures to DDoS attacks for RINA networks will be explored in the scenarios
with DDoS-specific policies. We anticipate the following ones:
• Isolation of flow allocation traffic. Flow Allocation traffic and RIB Daemon traffic can beisolated on different EFCP flows so that a DDoS attack does not impact DIF operations.
• Rate-limiting of flow allocation requests. In each system rate-limit Flow Allocation re-quests; in the directories (DIF Allocator or Flow Allocator) rate-limit the number of flowallocation requests that come from the same IPCP and/or are directed to the same IPCP.Analysing the pattern of proper flow allocate request distribution vs. the one in DDoS attacksis also key, then the DIF Allocator/ Flow Allocator can decide to block it.
• Resource allocator mitigation techniques. Resource allocators on IPCPs can collaborateto identify the traffic of the attack (based on observing the use of resources in multiple pointsat the DIF) and request IPCPs in provider routers close to the sources to discard the PDUsbelonging to the flows of the attackers.
6.4 Metrics and KPIs
The objective of this experiment is to compare the resiliency of RINA networks against DDoSattacks with that of current networks.
KPI Metric Current state of the art ARCFIRE Objective
Difficulty in discoveringvulnerable applications
Size of application names-pace
32 (IPv4) or 128 bits (IPv6)of address space * 16 bits oflocal ports per host
Size can be designed accord-ing to different goals (e.g. tomake scanning hard)
Cost of preventing vulnera-ble application discovery
Number of rules that need tobe written in access controlsystem
Number of firewall rulesdepends on the specificscenario
At least 50% less than in theTCP/IP case
Accuracy in detecting DDoSattack
Factors that bring to falsepositives or false negatives
Depends on particular strat-egy used
Using comparable strategies,its accuracy should be higheron RINA networks
Timeliness Delay in detection, response Depends on particular strat-egy used
Using comparable strategies,detection time should belower on RINA networks
Scalability Extra state in the networkdue to DDoS defense perattacker
Depends on particular strat-egy used
Using comparable strategies,the amount of state in thenetwork per attacker shouldbe lower
Table 29: KPIs for DDoS attacks
The KPIs and metrics that will be taken into consideration for Experiment 4 are listed inTable 29. The experiment will analyse the complexity of preparing a DDoS attack, which mainly
85
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
involves assembling an army of infected computers running bots that can be commanded to attackthe target(s). We will focus on the analysis of network-related issues. We will assume thatthe identification of vulnerable computers is carried out using network scanning techniques. Inaddition to measuring the difficulty in discovering vulnerable applications, we will also analyse theeffectiveness and complexity of the countermeasures that can be utilised to prevent the attackerto gain access to such vulnerable computers. The other KPIs focus on DDoS attack detection andmitigation. Many of the metrics that can be applied to evaluate the effectiveness of a DDoS solutionsuch as attack detection accuracy, precision, delay in detection/response, performance, etc. dependon the particular DDoS mitigation technique as well as on the architecture on which it is applied(TCP/IP or RINA). The main interest is to understand how RINA relates to TCP/IP when differentRINA strategies are deployed. Some of the time, a direct comparison of measurement data willnot provide a complete picture as it may measure inefficiencies in the implementation rather thanthe potential benefits of the RINA architecture. For instance, as introduced in section 6.1.2, themost effective solutions to fight DDoS attacks (the hybrid ones) require coordination by multiplesystems within different networks. In the current Internet this coordinated response is hard torealise because it requires the deployment of functions that facilitate this coordination (frameworkssuch as DEFCOM, for instance). We will envestigate how the comprehensive layer managementarchitecture can simplify the extra functions required for coordinated DDoS defense. The KPIs andmetrics that will be taken into account for this part of Experiment 4 are listed in Table 29.
6.5 Software
The required software for this experiment is summarised in Table 30.
• IRATI implementation. It will be used to implement multiple RINA-enabled systems andrun a variety of DIFs according to the experiment configuration depicted in Section 5.6.However, some specific policies to defend against certain forms of DDoS attacks may beimplemented by WP3 and contributed to the IRATI repository (such policies include forinstance the ability to rate-limit flow allocation requests).
• Nginx web server. A version of the Nginx web server ported to the RINA API will be usedas one of the target applications to attack.
• DIF Allocator. The prototype DIF Allocator that is under development within WP3 will beused to experiment with DDoS attacks involving applications in multiple DIFs. WP4 willprovide feedback to WP3 in regards to specific DIF Allocator features that may need to beimplemented for the experiment (such as rate-limiting flow allocation requests).
• DDoS botnet application. A prototype botnet application will be developed to carry on thedetection of vulnerable systems (recruiting zombies for the attack) and to perform a numberof different DDoS attacks (flow allocation request flooding, traffic flooding). The DDoS
86
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
application will be modelled after the source code of the Mirai botnet [31], adapting it to thecapabilities and restrictions of RINA networks.
• Demonstrator. The demonstrator will be used to test the correct behaviour of the softwarebefore deploying it on FED4FIRE+ testbeds. The demonstrator allows the experimenter tocarry out multiple experiment runs in a controlled environment with enough scale (up to 100RINA-enabled Virtual Machines).
• Rumba. Rumba is the experimentation and measurement framework tailored to RINAexperiments. It allows the definition of complex experiments involving multiple testbeds,being able to interact with testbed federation/automation tools such as jFed.
Software package Description License
IRATI [5] RINA Implementation for Linux OS GPL and LGPL
Nginx [32] Open source web server Modified 2-clause BSD
DIF Allocator [5] Locates applications over a variety of DIFs and collaborates withNMS(s) to create new DIFs
TBD
DDoS botnet application [5] Botnet for RINA networks that infects vulnerable systems andlaunches DDoS attacks
TBD
Demonstrator [7] Tool to setup self-contained VM-based IRATI testbeds GPL
Table 30: Software required for Experiment 4
6.6 Testbeds
Experiment 4 does not plan to use wireless resources, therefore the Virtual Wall will be the maintestbed of reference. The Virtual Wall can provide access to bare metal hardware to run the RINAimplementation and the applications required for the experiment. The number of resources availableat the Virtual Wall is also adequate (the largest scenario planned for the DDoS experiments containsabout 100 systems). After having run the experiment at the Virtual Wall, we plan to use a secondtestbed to enhance the geographical scope of the tests as well as the number of resources used.The current plan is to use some of the racks available through GENI, leveraging the expertise ofBoston University in using this testbed. However, the final decision will be made at the end of year2 (month 24), since inter-testbed connectivity is expected to change and evolve as the FED4IRE+project progresses its work. A summary of the testbeds that will be used from the selection made inD4.2 [9] can be seen in Table 31.
6.7 Planning
Table 6.7 shows the planned milestones for Experiment 4.
87
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
Testbed Purpose
Virtual Wall Provides access to enough bare metal servers to carry out DDoS experiments
GENI racks Enhance geographical diversity of the experiment; enable larger experiments
Table 31: Testbeds used in experiment 4
Milestone Month Description
MS 1 M18 All software for experiments with scenario 1 works on the Demonstrator, initial results using small-scale deployment
MS 2 M19 All software for experiments with scenario 1 works on the Demonstrator, initial results using large-scale deployment
MS 3 M22 Scenario 1 experiments carried out at VWall testbed using small-scale deployment (maps to ARCFIREproject milestone MS8)
MS 4 M23 Scenario 1 experiments carried out at VWall testbed using large-scale deployment
MS 5 M26 Scenario 2 experiments carried out at VWall testbed using both small-scale and large-scale deploy-ments
MS 6 M28 Multi-testbed experiment (tentative ones are VWall and GENI) carried out using large-scale deploy-ment (maps to ARCFIRE project milestone MS9)
Table 32: Milestones for DDoS experiments
88
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
7 Conclusion
This deliverable details the experimentation plans for ARCFIRE to be conducted during the secondpart of the project. Four main experiments were drafted, spanning different aspects such asmanageability, scalability, robustness and efficiency, with relevant KPIs identified for each of theseaspects per use case. On the basis of the use case, testbeds were selected for each experimentand the necessary software to execute the experiments were identified. A planning for eachof the experiments has been document to assess progress. As such, this document will guidethe experimenters to achieve meaningful results and aid them towards a successful and timelyconclusion.
89
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
References
[1] ARCFIRE consortium. (2016, September) H2020 arcfire deliverable d2.1: Convergednetwork operational environment analysis report. [Online]. Available: http://ict-arcfire.eu
[2] ——. (2016, December) H2020 arcfire deliverable d2.2: Converged service provider networkdesign report. [Online]. Available: http://ict-arcfire.eu
[3] International Telecommunication Unit, “Management framework for open systemsinterconnection (osi) for ccitt applications,” ITU-T Recommendation X.700 (09/92), March1992. [Online]. Available: www.itu.int/rec/T-REC-X.700
[4] L. Andrey, O. Festor, A. Lahmadi, A. Pras, and J. Schonwalder, “Survey of snmp performanceanalysis studies,” International Journal of Network Management, vol. 19, pp. 527–548, 2009.
[5] IRATI Github site. [Online]. Available: https://github.com/irati/stack
[6] Rina traffic generator. [Online]. Available: https://github.com/IRATI/traffic-generator
[7] Rina demonstrator github site. [Online]. Available: https://github.com/IRATI/demonstrator
[8] Rumba measurement framework. [Online]. Available: https://gitlab.com/arcfire/rumba
[9] ARCFIRE consortium. (2016, December) H2020 ARCFIRE deliverable D4.2: Experimentalinfrastructure available for experimentation report. [Online]. Available: http://ict-arcfire.eu
[10] V. Ishakian, J. Akinwumi, F. Esposito, and I. Matta, “On supporting mobility and multihomingin recursive internet architectures,” Computer Communications, vol. 35, no. 13, pp. 1561–1573, 2012.
[11] F. Giust, L. Cominardi, and C. Bernardos, “Distributed mobility management for future5g networks: overview and analysis of existing approaches,” IEEE IEEE CommunicationsMagazine, vol. 53, no. 1, pp. 142–149, January 2015.
[12] PRISTINE Consortium. (2016, June) FP7 PRISTINE, deliverable D4.3: Final specificationand consolidated implementation of security and reliability enablers. [Online]. Available:http://ict-pristine.eu
[13] E. C. Bormann, “Robust header compression (rohc): Framework and four profiles: Rtp, udp,esp, and uncompressed,” IETF Standards track, RFC 3095, 2001.
[14] (2017, Feb.) The IRATI ioq3 port. [Online]. Available: https://github.com/irati/ioq3
[15] ARCFIRE consortium. (2016, December) H2020 ARCFIRE deliverable D3.1: Integratedsoftware ready for experiments: RINA stack, Management System and measurementframework. [Online]. Available: http://ict-arcfire.eu
90
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
[16] OpenAIRInterface. [Online]. Available: http://www.openairinterface.org/
[17] D. Shinghal, M. Kunapareddy, V. Chetlapalli, V. B. James, and N. Akhtar, “Lte-advanced:Handover interruption time analysis for imt-a evaluation,” in 2011 International conferenceon Signal Processing, Communication, Computing and Networking Technologies(ICSCCN),2011.
[18] rlite github site. [Online]. Available: https://github.com/vmaffione/rlite
[19] jFed website. [Online]. Available: http://jfed.iminds.be
[20] Quagga routing suite. [Online]. Available: http://www.nongnu.org/quagga/
[21] S. Leon, J. Perello, D. Careglio, E. Grasa, M. Tarzan, N. Davies, and P. Thompson., “Assuringqos guarantees for heterogeneous services in rina networks with δq,” Proceedings of the 6thWorkshop on Network Infrastructure Services as part of Cloud Computing (NetCloud), 2016.
[22] B. Carpenter, R. Atkinson, and B. Flink, “Renumbering still needs work,” IETF Informationaldraft, RFC 5887, May 2010.
[23] D. Leroy and O. Bonaventure, “Preparing network configurations for renumbering,” Inter-national Journal of Network Management, vol. 19, no. 5, pp. 415–426, September/October2009.
[24] C. Buyukkoc. (2016, Mar.) Edge definition and how it fits with5g era networks. [Online]. Available: http://sdn.ieee.org/newsletter/march-2016/edge-definition-and-how-it-fits-with-5g-era-networks
[25] Verisign. (2016, October) Verisign distributed denial of service trends report, q3 2016.[Online]. Available: https://www.verisign.com/assets/report-ddos-trends-Q32016.pdf
[26] B. Krebs. (2015, September) Krebs on security hit by record ddos. [Online]. Available:https://krebsonsecurity.com/2016/09/krebsonsecurity-hit-with-record-ddos/
[27] A. Technica. (2016, October) How one rent-a-botnet army of cameras, dvrs causedinternet chaos. [Online]. Available: http://arstechnica.com/information-technology/2016/10/inside-the-machine-uprising-how-cameras-dvrs-took-down-parts-of-the-internet/
[28] P. Ferguson and D. Senie, “Network ingress filtering: Defeating denial of service attackswhich employ ip source address spoofing,” IETF Network Working Group, Best CurrentPractice 38, May 2000.
[29] S. T. Zargar, J. Joshi, and D. Tipper, “A survey of defense mechanisms against distributeddenial of service (ddos) flooding attacks,” IEEE communications surveys and tutorials, vol. 15,no. 4, pp. 2046–2069, 2013.
91
DRAFT
D4.3: Design of experimentalscenarios, selection of metricsand KPIs
Document: ARCFIRE D4.3
Date: February 17, 2017
[30] P. Teymoori, M. Welzl, S. Gjessing, E. Grasa, R. Riggio, K. Rausch, and D. Siracussa,“Congestion control in the recursive internetwork architecture (rina),” IEEE ICC 2016, NextGeneration Networking and Internet Symposium, 2016.
[31] (2016) Mirai botnet source code. [Online]. Available: https://github.com/jgamblin/Mirai-Source-Code
[32] Nginx website. [Online]. Available: http://hg.nginx.org/nginx.org
92