d4.2 final release of the service validation sdk toolset · 1.2.1 cloud-native support cloud-native...
TRANSCRIPT
D42 Final release of the service validation SDKtoolset
Project Acronym 5GTANGOProject Title 5G Development and Validation Platform for Global Industry-Specific Network
Services and AppsProject Number 761493 (co-funded by the European Commission through Horizon 2020)Instrument Collaborative Innovation ActionStart Date 01062017Duration 30 monthsThematic Priority H2020-ICT-2016-2017 ndash ICT-08-2017 ndash 5G PPP Convergent Technologies
Deliverable D42 Final release of the service validation SDK toolsetWorkpackage WP4Due Date M24Submission Date 05062019Version 10Status To be approved by ECEditor Wouter Tavernier (IMEC)Contributors Marios Touloupou (UPRC) Evgenia Kapassa (UPRC) Michael Filippakis
(UPRC) Dimitris Dres (UPRC) Manuel Peuster (UPB) Stefan Schneider(UPB) Eleni Fotopoulou (UBITECH) Anastasios Zafeiropoulos (UBITECH)Anton Roman Portabales Ana Pol Gonzalez (QUO) Peer Hasselmeyer (NEC)Thomas Soenen (IMEC) Askhat Nuriddinov (IMEC) Wouter Tavernier(IMEC)
Reviewer(s) Peer Hasselmeyer (NEC) Anton Roman Portabales (QUO) Sonia Castro(ATOS)
Keywords
SDK NFV development network softwarization testing
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Deliverable Type
R DocumentDEM Demonstrator pilot prototype XDEC Websites patent filings videos etcOTHER
Dissemination Level
PU Public XCO Confidential only for members of the consortium (including the Commission Ser-
vices)
DisclaimerThis document has been produced in the context of the 5GTANGO Project The research leading to these resultshas received funding from the European Communityrsquos 5G-PPP under grant agreement n 761493All information in this document is provided ldquoas isrdquo and no guarantee or warranty is given that the informationis fit for any particular purpose The user thereof uses the information at its sole risk and liabilityFor the avoidance of all doubts the European Commission has no liability in respect of this document which ismerely representing the authorsrsquo view
ii Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Executive Summary
The 5GTANGO SDK bundles a number of components to help the NFV service developer in thedevelopment process The SDK is the result of several iterations of development and evaluationThe SDK originated in the SONATA project 5GTANGO re-architected the resulting toolset andassociated workflow resulting into Release 41 which was documented in D41 In the final releasethe SDK has been revisited with respect to NFV trends and in the light of three 5GTANGO pilotsi) the communications pilot ii) the immersive media pilot and iii) the smart manufacturing pilot
The foundation of the SDK is based on a number of project management and coordinationtools in order to set up a local environment project and integrated portal environment to triggerthe other individual SDK tools To overcome tedious and error-prone generation of descriptorsa generator tool is able to automatically create descriptors based on a number of assumptionsTo overcome the limitations of virtual-appliance based services support for cloud native VNFsand associated scaling failover and associated state management has been integrated in tools forvalidation service specific manager creation and emulation The developer is assisted through adecision support engine which is able to automatically suggest related tests and VNFs based onthe profile of the developer Next these can be manually modified to accommodate the needs ofthe service Multi-platform support beyond 5GTANGO is enabled in the packaging tool whichis able to generate OSM as well as ONAP packages while the emulator is able to work as VIM forthe OSM platform (as well as for the 5GTANGO SP)
In order to thoroughly support real-world use cases the SDK as provided by Release 50 isguided by the three pilots and its relation to the VampV and Service Platform The SDK workflowhas therefore been reconsidered with respect to its support for functional and performance testingof individual VNFs as well as composed services The SDK provides three levels of local testingbefore the resulting package(s) are handed over to third-party testing as supported by the VampVandor SP The first level consists of (mainly) syntactical testing based on customisable validationof descriptor files Next ad-hoc functional testing is made possible through the deployment ofservices in the local emulator environment As many of these tests need to be executed after everydevelopment iteration (eg regression testing) a test framework enables scripting and automatedexecution of functional tests in a Python-based environment The third level of testing enablesto evaluate the performance of developed services through benchmarking them in a wide rangeof scenarios producing broad data sets The given tests subsequently can be analysed throughadvanced correlation and time series analysis tools of the analytics engine
The 5GTANGO SDK is here to stay The workflow it provides as well as its set of novel toolsand their relationship and interfaces is documented in a self-contained manner in this deliverableHowever the true sustainability is enabled through the corresponding open-source repositories anddocumentation in public GitHub repositories GitHub provides well-known mechanisms to reportissues and contribute via pull requests to future enhancements of the SDK tool set Proposals forfuture extensions of SDK functionality beyond 5GTANGO are provided and fuel our future hopsfor the trendsetting work that has been delivered through Release 50 of the 5GTANGO SDK
5GTANGO Public iii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Contents
List of Figures vii
List of Tables ix
1 Introduction 111 Document scope 1
12 Overview 1
121 Cloud-native support 1
122 Validation verification and testing 2
123 Extensible design and support for alternate platforms 2
13 Document structure 2
2 5GTANGO Development and Testing Lifecycle 321 Phase 1 Development and Testing using the SDK 3
22 Phase 2 Validation and Verification using the VampV Platform 5
23 Phase 3 Deployment and Execution using the Service Platform 5
3 Architecture 731 Components 8
311 Schema for Descriptors 8
312 SDK Portal 10
313 Decision Support Engine 12
314 Descriptor generator and project management 15
315 Packager 20
316 Validator 26
317 Testing framework 29
318 Development and testing of specific manager functionality 32
319 State management support 36
3110 Emulator 40
3111 Benchmarker 43
3112 Analytics Engine 49
32 External Interfaces 53
321 Components with external interfaces 53
322 5GTANGOrsquos advanced package format as main interface 53
4 Final release features 54
5 Pilot Requirements 5551 Communications Pilot 56
52 Immersive Media Pilot 56
53 Smart Manufacturing Pilot 56
iv Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution 5861 Descriptors schema generation packaging and validation 5862 Development workflow and portal 5963 Local testing and analysis 59
7 Source Code and installation 6071 Installation 60
8 Conclusions 61
A Bibliography 62
5GTANGO Public v
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Deliverable Type
R DocumentDEM Demonstrator pilot prototype XDEC Websites patent filings videos etcOTHER
Dissemination Level
PU Public XCO Confidential only for members of the consortium (including the Commission Ser-
vices)
DisclaimerThis document has been produced in the context of the 5GTANGO Project The research leading to these resultshas received funding from the European Communityrsquos 5G-PPP under grant agreement n 761493All information in this document is provided ldquoas isrdquo and no guarantee or warranty is given that the informationis fit for any particular purpose The user thereof uses the information at its sole risk and liabilityFor the avoidance of all doubts the European Commission has no liability in respect of this document which ismerely representing the authorsrsquo view
ii Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Executive Summary
The 5GTANGO SDK bundles a number of components to help the NFV service developer in thedevelopment process The SDK is the result of several iterations of development and evaluationThe SDK originated in the SONATA project 5GTANGO re-architected the resulting toolset andassociated workflow resulting into Release 41 which was documented in D41 In the final releasethe SDK has been revisited with respect to NFV trends and in the light of three 5GTANGO pilotsi) the communications pilot ii) the immersive media pilot and iii) the smart manufacturing pilot
The foundation of the SDK is based on a number of project management and coordinationtools in order to set up a local environment project and integrated portal environment to triggerthe other individual SDK tools To overcome tedious and error-prone generation of descriptorsa generator tool is able to automatically create descriptors based on a number of assumptionsTo overcome the limitations of virtual-appliance based services support for cloud native VNFsand associated scaling failover and associated state management has been integrated in tools forvalidation service specific manager creation and emulation The developer is assisted through adecision support engine which is able to automatically suggest related tests and VNFs based onthe profile of the developer Next these can be manually modified to accommodate the needs ofthe service Multi-platform support beyond 5GTANGO is enabled in the packaging tool whichis able to generate OSM as well as ONAP packages while the emulator is able to work as VIM forthe OSM platform (as well as for the 5GTANGO SP)
In order to thoroughly support real-world use cases the SDK as provided by Release 50 isguided by the three pilots and its relation to the VampV and Service Platform The SDK workflowhas therefore been reconsidered with respect to its support for functional and performance testingof individual VNFs as well as composed services The SDK provides three levels of local testingbefore the resulting package(s) are handed over to third-party testing as supported by the VampVandor SP The first level consists of (mainly) syntactical testing based on customisable validationof descriptor files Next ad-hoc functional testing is made possible through the deployment ofservices in the local emulator environment As many of these tests need to be executed after everydevelopment iteration (eg regression testing) a test framework enables scripting and automatedexecution of functional tests in a Python-based environment The third level of testing enablesto evaluate the performance of developed services through benchmarking them in a wide rangeof scenarios producing broad data sets The given tests subsequently can be analysed throughadvanced correlation and time series analysis tools of the analytics engine
The 5GTANGO SDK is here to stay The workflow it provides as well as its set of novel toolsand their relationship and interfaces is documented in a self-contained manner in this deliverableHowever the true sustainability is enabled through the corresponding open-source repositories anddocumentation in public GitHub repositories GitHub provides well-known mechanisms to reportissues and contribute via pull requests to future enhancements of the SDK tool set Proposals forfuture extensions of SDK functionality beyond 5GTANGO are provided and fuel our future hopsfor the trendsetting work that has been delivered through Release 50 of the 5GTANGO SDK
5GTANGO Public iii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Contents
List of Figures vii
List of Tables ix
1 Introduction 111 Document scope 1
12 Overview 1
121 Cloud-native support 1
122 Validation verification and testing 2
123 Extensible design and support for alternate platforms 2
13 Document structure 2
2 5GTANGO Development and Testing Lifecycle 321 Phase 1 Development and Testing using the SDK 3
22 Phase 2 Validation and Verification using the VampV Platform 5
23 Phase 3 Deployment and Execution using the Service Platform 5
3 Architecture 731 Components 8
311 Schema for Descriptors 8
312 SDK Portal 10
313 Decision Support Engine 12
314 Descriptor generator and project management 15
315 Packager 20
316 Validator 26
317 Testing framework 29
318 Development and testing of specific manager functionality 32
319 State management support 36
3110 Emulator 40
3111 Benchmarker 43
3112 Analytics Engine 49
32 External Interfaces 53
321 Components with external interfaces 53
322 5GTANGOrsquos advanced package format as main interface 53
4 Final release features 54
5 Pilot Requirements 5551 Communications Pilot 56
52 Immersive Media Pilot 56
53 Smart Manufacturing Pilot 56
iv Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution 5861 Descriptors schema generation packaging and validation 5862 Development workflow and portal 5963 Local testing and analysis 59
7 Source Code and installation 6071 Installation 60
8 Conclusions 61
A Bibliography 62
5GTANGO Public v
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Executive Summary
The 5GTANGO SDK bundles a number of components to help the NFV service developer in thedevelopment process The SDK is the result of several iterations of development and evaluationThe SDK originated in the SONATA project 5GTANGO re-architected the resulting toolset andassociated workflow resulting into Release 41 which was documented in D41 In the final releasethe SDK has been revisited with respect to NFV trends and in the light of three 5GTANGO pilotsi) the communications pilot ii) the immersive media pilot and iii) the smart manufacturing pilot
The foundation of the SDK is based on a number of project management and coordinationtools in order to set up a local environment project and integrated portal environment to triggerthe other individual SDK tools To overcome tedious and error-prone generation of descriptorsa generator tool is able to automatically create descriptors based on a number of assumptionsTo overcome the limitations of virtual-appliance based services support for cloud native VNFsand associated scaling failover and associated state management has been integrated in tools forvalidation service specific manager creation and emulation The developer is assisted through adecision support engine which is able to automatically suggest related tests and VNFs based onthe profile of the developer Next these can be manually modified to accommodate the needs ofthe service Multi-platform support beyond 5GTANGO is enabled in the packaging tool whichis able to generate OSM as well as ONAP packages while the emulator is able to work as VIM forthe OSM platform (as well as for the 5GTANGO SP)
In order to thoroughly support real-world use cases the SDK as provided by Release 50 isguided by the three pilots and its relation to the VampV and Service Platform The SDK workflowhas therefore been reconsidered with respect to its support for functional and performance testingof individual VNFs as well as composed services The SDK provides three levels of local testingbefore the resulting package(s) are handed over to third-party testing as supported by the VampVandor SP The first level consists of (mainly) syntactical testing based on customisable validationof descriptor files Next ad-hoc functional testing is made possible through the deployment ofservices in the local emulator environment As many of these tests need to be executed after everydevelopment iteration (eg regression testing) a test framework enables scripting and automatedexecution of functional tests in a Python-based environment The third level of testing enablesto evaluate the performance of developed services through benchmarking them in a wide rangeof scenarios producing broad data sets The given tests subsequently can be analysed throughadvanced correlation and time series analysis tools of the analytics engine
The 5GTANGO SDK is here to stay The workflow it provides as well as its set of novel toolsand their relationship and interfaces is documented in a self-contained manner in this deliverableHowever the true sustainability is enabled through the corresponding open-source repositories anddocumentation in public GitHub repositories GitHub provides well-known mechanisms to reportissues and contribute via pull requests to future enhancements of the SDK tool set Proposals forfuture extensions of SDK functionality beyond 5GTANGO are provided and fuel our future hopsfor the trendsetting work that has been delivered through Release 50 of the 5GTANGO SDK
5GTANGO Public iii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Contents
List of Figures vii
List of Tables ix
1 Introduction 111 Document scope 1
12 Overview 1
121 Cloud-native support 1
122 Validation verification and testing 2
123 Extensible design and support for alternate platforms 2
13 Document structure 2
2 5GTANGO Development and Testing Lifecycle 321 Phase 1 Development and Testing using the SDK 3
22 Phase 2 Validation and Verification using the VampV Platform 5
23 Phase 3 Deployment and Execution using the Service Platform 5
3 Architecture 731 Components 8
311 Schema for Descriptors 8
312 SDK Portal 10
313 Decision Support Engine 12
314 Descriptor generator and project management 15
315 Packager 20
316 Validator 26
317 Testing framework 29
318 Development and testing of specific manager functionality 32
319 State management support 36
3110 Emulator 40
3111 Benchmarker 43
3112 Analytics Engine 49
32 External Interfaces 53
321 Components with external interfaces 53
322 5GTANGOrsquos advanced package format as main interface 53
4 Final release features 54
5 Pilot Requirements 5551 Communications Pilot 56
52 Immersive Media Pilot 56
53 Smart Manufacturing Pilot 56
iv Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution 5861 Descriptors schema generation packaging and validation 5862 Development workflow and portal 5963 Local testing and analysis 59
7 Source Code and installation 6071 Installation 60
8 Conclusions 61
A Bibliography 62
5GTANGO Public v
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Contents
List of Figures vii
List of Tables ix
1 Introduction 111 Document scope 1
12 Overview 1
121 Cloud-native support 1
122 Validation verification and testing 2
123 Extensible design and support for alternate platforms 2
13 Document structure 2
2 5GTANGO Development and Testing Lifecycle 321 Phase 1 Development and Testing using the SDK 3
22 Phase 2 Validation and Verification using the VampV Platform 5
23 Phase 3 Deployment and Execution using the Service Platform 5
3 Architecture 731 Components 8
311 Schema for Descriptors 8
312 SDK Portal 10
313 Decision Support Engine 12
314 Descriptor generator and project management 15
315 Packager 20
316 Validator 26
317 Testing framework 29
318 Development and testing of specific manager functionality 32
319 State management support 36
3110 Emulator 40
3111 Benchmarker 43
3112 Analytics Engine 49
32 External Interfaces 53
321 Components with external interfaces 53
322 5GTANGOrsquos advanced package format as main interface 53
4 Final release features 54
5 Pilot Requirements 5551 Communications Pilot 56
52 Immersive Media Pilot 56
53 Smart Manufacturing Pilot 56
iv Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution 5861 Descriptors schema generation packaging and validation 5862 Development workflow and portal 5963 Local testing and analysis 59
7 Source Code and installation 6071 Installation 60
8 Conclusions 61
A Bibliography 62
5GTANGO Public v
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution 5861 Descriptors schema generation packaging and validation 5862 Development workflow and portal 5963 Local testing and analysis 59
7 Source Code and installation 6071 Installation 60
8 Conclusions 61
A Bibliography 62
5GTANGO Public v
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Figures
21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP 422 Components and steps in the SDK testing workflow 6
31 SDK workflow and tool overview 732 SDK Portal Architecture 1133 Workflow between the portal and the recommender 1434 Improved extensible architecture with modular generation plugins for different plat-
forms (eg 5GTANGO OSM or ONAP) 1635 Overview of the tng-sdk-project REST API 1936 Latest version of automatically generated OpenAPI documentation of REST API
endpoints 2437 PackagerValidator call graph for packaging example 2538 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced
5GTANGO package format 2539 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos
REST API endpoints 29310 tng-sdk-sm local setup for SSM testing 35311 State management patterns 37312 Lifecycle process migration 39313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smart
manufacturing pilot on top of the emulator [39] 43314 High-level architecture artifacts and workflows [34] 44315 Part of the YANG model specifying the format of the performance experiment de-
scriptors (PED) 46316 Prometheus dashboard showing the execution of multiple experiment rounds 48317 Example of a time series metric recorded during a single experiment round 48318 Profiling Sequence 50319 Correlation analysis of Suricata VNF 51320 Correlation results in table format 52321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric 52
5GTANGO Public vii
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
List of Tables
41 Final release SDK highlights (new components in bold) 54
51 Pilot Requirements vs Final Release Features 55
5GTANGO Public ix
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
1 Introduction
11 Document scope
The objective of this Deliverable D42 is to describe the design and implementation details of thelast release (SONATA 50) of the 5GTANGO SDK due in project month 24 (M24 May 2019) Thedescription contained in this document is an update of Deliverable D41 [20] released in month10 (M10 March 2018) The latter focused on the first 5GTANGO-powered release (SONATA40) while it introduced the overall workflow and the core components of the 5GTANGO SDK incomparison to those of SONATA For this release we maintain the overall structure however asignificant number of components and features were added to further improve the SDK Particularattention has been paid to the sustainability and independence of the toolset as well as other(MANO) platforms (eg OSM and ONAP) in addition to the 5GTANGO Service Platform Whereneeded core architectural aspects have been repeated in order to make the document as self-contained as possible
12 Overview
The SDK has undergone different evolutions since its initial birth in the SONATA project [7] Thefirst 5GTANGO release (SONATA Release 40) of the SDK was described in D41 and focusedon opening the toolset towards other NFV initiatives beyond the initially focused SONATA and5GTANGO platforms
The SDK was thoroughly extended and refined in mind of these other initiatives embracing newtrends in NFV (eg cloud-native VNFs) and providing optimal support for the different 5GTANGOpilots As from the very beginning this final version is released as open source making it publiclyavailable for a large community (Github)
Recent trends in NFV are characterized through the realization that traditional virtual-appliancebased VNFs (NFs implemented as virtual machines) do not scale well and defeat the originalobjectives of agility and resource efficiency of NFV Below we stress three main areas on which theSDK was refined
121 Cloud-native support
Cloud-native design of services and NFs are therefore considered as the anticipated future of NFVCloud-native design focuses on small components (ideally containers) and appropriate managementto support dynamic provisioning scaling and failover of services and associated components OurSDK components already anticipated this evolution in the prior release through the availability ofa container-focused emulator and further strengthened its position by providing extended cloud-native support in a range of other tools Schema were extended to support CNFs and cloud-nativedeployment units The SDK validation was extended to inspect and validate associated cloud-nativesyntax and where needed support associated customized rules In addition the SSMFSM creationand state management frameworks have been extended in order to further ease the development of(cloud-native) scaling and recovery mechanisms
5GTANGO Public 1
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
122 Validation verification and testing
Validation Verification and Testing become of ever-growing importance in the modern NFV land-scape Indeed software-based components and services are now rapidly replacing hardware-basedfunctionalities In order to profit from quicker development times and shorter times to marketthe NFV toolkit needs to support solid and rapid testing mechanisms This release builds furtheron foundations of the existing SDK As a result the SDK has now a well-rounded set of featuressupporting i) generation of descriptors with limited failures ii) validation of descriptors iii) (ad-hoc) emulation of services and components iv) development of (Python-based) tests which can beexecuted in a fully automated way on the local PC of the developer and seamlessly reused on third-party VampV platforms and v) generation and analysis of performance data of services through theSDK benchmarker and analytics engine In addition a recommender system has been introducedto assist developers to select already existing tests into their testing workflow
123 Extensible design and support for alternate platforms
In the last years 5GTANGO has grown towards a major MANO platform and SDK providerfor NFV deployments In addition to the trendsetting features supporting customised MANO-workflows through SSMs extensive slice support and advanced SDK functionality including theOSM-adopted emulator our SDK also aims to be future proof through an extensible design andsupport for alternate platforms As a result the SDK packaging tool supports OSM ONAP and5GTANGO packages and can be further extended towards other platforms in the future Theemulator has been extended to support the OSM and 5GTANGO MANO platform and its opendesign supports seamless integration of others Most of the SDK components have well-definedand stable CLI interfaces but some of them have REST APIs available making them suitable forbeing used as a service in the context of other platforms The analytics engine now has fine-grainedsupport for the output produced by either the SDK benchmarking tool or the monitoring data asproduced by the monitoring components part of the service platform and VampV however the broadPrometheus support and open design makes them suitable candidates for reuse in other contexts
These three areas in relationship to the different 5GTANGO pilots have steered the design anddevelopment of the latest release of the SDK The coming sections should be read from this per-spective and will provide further details on their primary aims their use their mutual relationshipand their relationships to the pilots
13 Document structure
The rest of the document is structured as follows In [Sec 2] we document the 5GTANGO philos-ophy on testing from an SDK perspective and put it into relation to the test handling as providedby the 5GTANGO VampV in WP3 [Sec 3] provides the core of the document by providing anoverview of the extended SDK its improved workflow and associated processes followed by anin-depth documentation of the individual components [Sec 32] details the interfaces of SDK com-ponents towards other 5GTANGO parts as well as the interfaces used between the individual SDKcomponents [Sec 4] provides a condensed overview of the highlights of Release 50 of the SDKIn [Sec 5] we clarify the role of different pilots on the developed SDK tools and vice versa Theremaining [Sec 6] and [Sec 7] provide pointers for the community in order to facilitate contributionto the SDK and associated source code repositories Finally [Sec 8] provides concluding thoughtsand pointers for future work beyond the term of the project
2 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
2 5GTANGO Development and Testing Lifecycle
The increased level of programmability as enabled by SDNNFV technology is one of the keyenablers of 5G technology [43] 5GTANGO fully embraces the ldquosoftwarizationrdquo of communicationservices and adopts a DevOps approach that enables rapid and seamless interactions between servicedevelopment and its use in production systems Testing validation and verification ensure thatoperators and service users can be sure that VNFs and associated Network Services behave in astable reproducible and expected manner
5GTANGO uses a three-phased approach consisting of i) Development ii) Verification amp Val-idation and iii) Production Functionality in support of testing impacts all three phases Thefirst phase focuses on ad-hoc testing in a local environment together with the development andlocal execution of automated test scripts and associated probes The second phase focuses on theexecution test scripts and probes on a range of different environments and conditions Phase 3uses testing and monitoring in production environments and relies on developed tests to assess anddebug failure scenarios
In the following subsections each of the three phases and their role in the testing lifecycle willbe described in more detail
21 Phase 1 Development and Testing using the SDK
The first phase of the adopted DevOps approach consists of VNF and service development assupported by the 5GTANGO SDK toolset (Fig 22) All SDK-based development is based onthe implementation of individual VNFs (step 1) As documented in later sections the major goalof the SDK is to assist in the service composition test implementation and local testing of NFVservices and comprising VNFs The individual tools and workflow are described in the next sectionhowever here we will highlight the role of these tools with respect to testing
Given the individual implementations the SDK provides the functionality to set up the projectenvironment (step 2) and actually prepare the corresponding descriptors for the network service andVNFs (step 3) Once all individual descriptors are prepared the packaging tool produces onboard-abledeployable packages (step 4) which are syntactically validated using the tng-sdk-validationtool (step 5) Local ad-hoc testing is made possible through the vim-emu component enabling de-velopers to quickly interact with locally deployed services In parallel scripted (functional) testscan be developed and locally executed through the tng-sdk-test and vim-emu components (step6) Contrary to ad-hoc testing scripted testing enables automated workflows including forms ofunit integration and regression tests to be executed after every implementation iteration Perfor-mance testing is prepared through the generation and evaluation of a range of benchmarking setupsas facilitated by the tng-sdk-benchmark tool (step 7) The resulting performance test data canbe analysed using the analytics engine (step 8)
The 5GTANGO DevOps vision aims to maximally support iterations in this development andtesting and deployment process The feedback produced by the testing tools might need changes inthe original implementation producing a novel setup to be tested Once all local testing has beenfinalized satisfactory testing and deployment can advance to the next phase by handing over thedeveloped components descriptors and tests to the VnV components Testing in phase 2 will then
5GTANGO Public 3
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 21 5GTANGOrsquos general testing workflow involving the SDK VnV and SP
4 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
enable to extend and re-evaluate service packages in a wide range of environments and resourceconfigurations compared to the local setup of the developer
22 Phase 2 Validation and Verification using the VampV Platform
After service developers have packaged and tested their services locally they still have significantwork to complete as they have only passed the ldquoentry levelrdquo stage of validation The secondphase of the 5GTANGO development lifecycle moves to a much deeper level of testing that allowsservice developers to fully validate that their services are ready to be deployed to production Thisldquovalidation and verificationrdquo lifecycle organizes tests in test plans (step 10) curates the tests andservices within packages (step 11) and finally executes the tests themselves (step 12) Test plansallow developers to associate network services with particular tests using descriptors (NetworkService Descriptor a template that describes the deployment of a Network Service includingservice topology Test Descriptor a template that describes the steps to setup exercise andverify executable tests for a service or VNF) All test results are saved to a test result repository(step 14) for storage and subsequent deeper analysis (step 15) The test executions themselvesgenerate important measurement data that is monitored by the platforms monitoring engine (step13) Monitoring data is in turn analyzed by the platformrsquos data analytics engine that allowsservice developers to profile their service in terms of its resource consumption efficiency and overallperformance
The platformrsquos recommendation engine can recommend service developers what next actionsto take to improve their service quality (step 9) These next actions or steps take the form ofadditional tests that should be executed in the VampV platform The overall verification lifecycle isflexible in that service developers may choose to only use the basic VampV lifecycle that of planningcurating and executing tests The other steps described in the lifecycle diagram above Fig 21 areoptional in the overall workflow
23 Phase 3 Deployment and Execution using the Service Platform
The above described steps of Phases I and II are crucial in the development or update of networkservices in the sense that they shorten the feedback to the service developer making himherimprove the service as early as possible
But some defects or malfunctions might only be detected when the service is already deployedin production eg those depending on the concrete production environment namely those relatedto performance or on inter-instances interactions To address also this part of the service lifecycle5GTANGO provides the ability for the developer of the service to request monitoring data fromone or more instances of the service developed
Therefore after a successful validation and verification done in Phase 2 the network service mustbe onboarded into the service platformrsquos catalogue which means uploading (step 16) un-packaging(step 17) validating (step 18) and storing (step 19) the packagersquos artifacts in the catalogue Lateron the Service Platform owner can include the onboarded service in a slice (step 20) which can belater instantiated (step 21a) Depending on the service it can also be instantiated without beingpart of any slice (step 21b)
The instantiation process communicates to the monitoring manager the monitoring parametersof the service (step 22) which starts receiving monitoring data from the service instance Theservice developer can then request that data for limited periods of time (step 23) thus closinganother (and the broadest one) feedback cycle
5GTANGO Public 5
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 22 Components and steps in the SDK testing workflow
6 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3 Architecture
Building further on the work documented in D41 the SDK architecture follows a GIT-alike archi-tecture in which a set of 14 independently usable tools and components can be combined in a usefulmanner in order to compose test and evaluate services which in a later phase can be deployed oneither the 5GTANGO VampV or on the SP (as documented in the prior sections) Each individualtool is designed to provide added value on its own and has potential usage beyond the initial scopeof 5GTANGO However the SDK was conceived in such a way that all the tools can also worktogether in a canonical manner enabling different forms of iterations of development and testingon the local machine of the developer
Fig 31 depicts this canonical workflow consisting of 6 major steps of development and coor-dinated by a number of overarching SDK environment tools These three coordination toolssupport the developer in preparing the local development environment through the creation ofworkspaces and project folders (tng-workspace and tng-project tools) and also includes a GUIportal which is able to quickly and accessibly trigger most of the available SDK tools
Every 5GTANGO development process is directed (first step) by the implementation of in-dividual VNFs specific managers and appropriate tests While 5GTANGO does not focus onthe individual VNF implementation tng-sdk-img provides support to convert docker images intoVMs therefore broadening the scope of VIMs on which VNFs can be deployed The implemen-tation of tests of VNFs services and specific managers (SMs) is supported by tng-sdk-test andtng-sdk-sm
Once the individual components are available the SDK provides a number of tools to actuallycompose (step 2) these components into services described through descriptors tng-sdk-schemadefines the descriptor formats that can be used while tng-sdk-descriptorgen provides usabletemplates to start from In addition tng-vnv-dsm provides recommendations on which componentscan be appropriately combined
Given the appropriate descriptors the entire service and collection of tests can be bundled amppackaged (step 3) using the tng-sdk-package tool supporting multiple deployment platformsbesides 5GTANGO SP
Any given package subsequently needs to be validated and tested (step 4) adequately beforeit should be used in production environments Syntactic validation and selected (static) semanticchecks of involved descriptors are supported through tng-sdk-validation On the other handmore extensive tests can only be performed if the service and its associated specific managers areactually deployed in a local environment The emulator vim-emu enables ad-hoc testing on the local
Figure 31 SDK workflow and tool overview
5GTANGO Public 7
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
developer laptop In addition test automation tools support extensive and repeatable execution ofpre-defined functional tests using tng-sdk-test and tng-sdk-sm
Obviously these first 4 steps may involve iterations as tests might reflect bugs which need to beaddressed possibly in the individual implementation (step 1) or composition (step 2) each of themrequiring subsequent packaging (step 3) before they can be retested (step 4)
Once one or multiple iterations of step 1 to step 4 as well as functional tests have been successfullyexecuted the next phase might consist of assessing the performance of the resulting service Thelatter involves two aspects i) producing the appropriate set of measurement data under a widenumber of environmental conditions (eg resource restrictions) which is called benchmarking(step 5) and ii) analyzing (step 6) the produced performance data with the purpose of identifyingperformance correlations andor bottlenecks
31 Components
311 Schema for Descriptors
Descriptors specify various artifacts in 5GTANGO They specify relevant metadata for VNFsnetwork services packages tests slices SLAs and policies The data included in the descriptorsis used by numerous 5GTANGO tools in the entire lifecycle from development via validation andverification to deployment with the service platform
To ensure that all these tools can work seamlessly with the descriptors consistent structureand syntax are crucial To this end 5GTANGO uses schemas to define required and optionalfields their type and structure for all descriptors While the descriptors and schemas themselvesare written in YAML the schema checking is implemented as a small validation tool using JSONschema (draft-04) [4]
Descriptors and schemas were introduced from the beginning of 5GTANGO and described alreadyin deliverable D41 [20] Since then the descriptors and schemas were evolved continuously In thelast release cycle more features were added to support several pilot requirements
3111 Release 50
Overview of changes since the last release
bull Support for new VNFD types
ndash Support for cloud native deployment units within VNFDs CNF (cloud native networkfunctions) support
ndash Support for physical deployment units within VNFDs PNF (physical network functions)support
ndash Support for mixed deployment units within one VNFD HNF (hybrid network functions)support
bull Allow specifying arbitrary QoS requirements for vLinks and connection points in VNFDs andNSDs
bull Support for multiple VM flavours of a VNF with different resource and QoS requirements
bull Allow specifying alternative VDU images for deploying VNFs on multiple architectures (egx86 ARM etc)
8 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Additional optional fields as requested by VNF vendors at the ETSI plug test vm flavorsecurity groups etc for each VDU
3112 Cloud-native Deployment Units
A current trend in NFV is the deployment of VNFs in lightweight containers (eg Docker [30])rather than heavy-weight VMs Such container-based VNFs are called ldquocloud-native VNFsrdquo cor-responding to the rise of new cloud container orchestrators like Kubernetes [5]
In 5GTANGO all three pilots aim to implement at least some VNFs as lightweight cloud-nativeVNFs to achieve faster deployment update and scaling times while requiring less resources Fordeployment of such cloud-native VNFs on Kubernetes additional information is required by theservice platform eg concerning ports for accessing the containers or additional environmentalvariables
The updated VNFD of release 50 support cloud-native deployment units (CDUs) as alternativesto typical virtual deployment units (VDUs) or physical deployment units (PDUs) The examplebelow shows the relevant VNFD section of a cloud-native VNF (CNF) that is being used in the5GTANGO smart manufacturing pilot In this CNF the ldquocloud connectorrdquo there are four con-tainers described by four CDUs that live alongside each other in one VNF The new CDUs allowto clearly specify container image connection points and environmental variables as required
CDUs of the cloud connector VNF in 5GTANGOrsquos smart manufacturing pilot
cloudnative_deployment_units
- id cdu01
image sonatanfvvnf-cc-brokerk8s
connection_points
- id int-mqtt
port 1883
- id cdu02
image sonatanfvvnf-cc-processork8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu03
image sonatanfvvnf-cc-mqttexporterk8s
connection_points []
parameters
env
MQTT_BROKER_HOST localhost
MQTT_BROKER_PORT 1883
- id cdu04
image sonatanfvvnf-cc-databasek8s
connection_points
- id int-prometheus
port 9090
5GTANGO Public 9
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3113 QoS Requirements and VM Flavours
Similar to CDUs being used in the smart manufacturing (and other) pilots further improvementsof the schemas were also driven by pilot requirements For example the real-time communicationpilot considers varying QoS requirements for different service levels such as ldquobronzerdquo ldquosilverrdquoldquogoldrdquo
To this end the new VNFD and NSD schemas support annotations for explicit QoS requirementsat VDUs virtual links or connection points These requirements can then be considered by theservice platform during orchestration and by the VIM eg OpenStack [37] The example belowshows the relevant section of a connection point annotated with QoS requirements
explicit QoS requirements (supported by OpenStack)
- id eth1
qos_requirements
bandwidth_limit
bandwidth 2
bandwidth_unit Mbps
minimum_bandwidth
bandwidth 0
bandwidth_unit kbps
Furthermore VNFDs and NSDs now support multiple flavours (eg bronze to gold) which candefine different resource requirements or QoS requirements This allows the service platform todynamically match and use the suitable flavour according to current SLAs
312 SDK Portal
The 5GTANGO SDK consists of numerous tools that are useful for different stages in the develop-ment workflow ranging from descriptor generation over validation to packaging of services readyto be deployed Most of these tools provide a command-line interface (CLI) and a microserviceREST API
The correct use of these tools in the correct order may be challenging for new users and requiresthe local setup and installation of all corresponding tools This can be a particular challenge forusers from vertical industries such as manufacturing
To this end 5GTANGO provides a graphical SDK portal that allows a seamless and effortlessdevelopment workflow This SDK portal can be hosted remotely and provided to new users withoutrequiring any local installation Hence it can lower the barrier to entry - especially as mentionedabove for domain experts from vertical domains
3121 Release 50
The SDK portal is a completely new component in release 50 It was not available in previousreleases
3122 Architecture
The SDK portal front-end leverages and reuses the same code basis as the graphical portal for theVampV and the SP This allows minimizing duplicate code improving the common code base easilyand ensures a consistent look and feel The shared code with the VampV and SP portal concernscommon front-end components and functionalities such as the general layout and the behavior ofthe sidebar menu
10 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 32 SDK Portal Architecture
Nevertheless the SDK portal will be used independently from the portal for the VampV and theSP The portal comes with a pre-defined configuration for building and deploying the SDK portal(or the VampV portal or the SP portal) effortlessly The front-end is written completely in Angular60 [3] to ensure a modern responsive look and behavior
Fig 32 shows the overall architecture of the SDK portal The front-end connects to and commu-nicates with the various SDK tools These SDK tools act as back-end running in Docker containerswhile they expose REST APIs
The tng-sdk-project (Sec 314) acts as main back-end container and handles the direct com-munication with the SDK portal It holds all generated descriptors NFV projects packages andcoordinates the communication with other SDK tools eg tng-sdk-validation (Sec 316) andtng-sdk-package (Sec 315) by calling their corresponding REST APIs and passing the responseto the SDK portal front-end
This central position of the tng-sdk-project in the architecture requires changes and extensionof its REST API and previous capabilities but it comes with considerable advantages It simplifiesthe front-endback-end communication since the front-end only has to interact directly with a singlecontainer It also enables consistent data handling storage and serving from the tng-sdk-projectrather than distributing and cluttering various relevant files (eg descriptors project manifestpackages ) across several containers and even storing them in the front-end itself Instead thetng-sdk-project holds all relevant files and exposes them via a simple REST API for externalaccess from the SDK portal
5GTANGO Public 11
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3123 Installation
As mentioned before the SDK portal front-end shares a common code base with the VampV and SPportal but can be deployed and used independently This split reflects the typical use case wheredevelopers use the SDK portals and testers and operators use the VampV and SP portal respectivelyIn the case that multiple of these roles are handled by the same people or organization a combinedSDK and VampV or SP portal could also be deployed
Deployment can be done locally simply by pulling images of the SDK portal along with the otherSDK tools from Docker Hub [29] Alternatively 5GTANGO or others can provide an already hostedversion that is accessible remotely (via a fixed URL) without any further installation overheadGenerated descriptors projects and packages could then be downloaded directly from the browser
3124 Usage
The usage of the SDK portal is simple as users just follow along the graphical workflow from onestep to another while filling in the required forms as indicated The SDK portal provides directfeedback of which information is required or whether information is missing or corrupt
The core features of the SDK portal are
bull Descriptor and project generation Generates new descriptors based on provided high-levelinformation and stores them in a newly created NFV project including the correspondingproject manifest
bull Validation Validation of generated descriptors to check their correctness in terms of syntaxand integrity
bull Packaging Packaging of the complete NFV project into a single package which can then beon-boarded to the VampV platform or the service platform
Envisioned advanced features of the SDK portal are
bull Editing of generated descriptors in an online web IDE
bull Project management After generating (and editing) descriptors for a new project add orremove further files
bull The validation tool could provide extensive graphical insight rather than simple passfailinformation
bull In addition to downloading created packages the SDK portal could offer direct on-boardingof packages to connected VampV or service platforms
313 Decision Support Engine
The Decision Support Engine or tng-vnv-dsm is a novel tool introduced into the 5GTANGO SDKin release 50 It is based on the ideas initially presented in one of the deliverables of 5GTANGO[41] while it aims to unburden the test developers from the selection of the tests that will checkthe functionality of their NSs The goal of the decision support engine is to automate the processof the test selection providing some recommendation preferences based on the users previousactivity Thus tng-vnv-dsm is a recommendation system that uses Collaborative Filtering methodswhich are based on collecting and analyzing large amounts of information on usersrsquo behaviorsactivities or preferences and predict what users will prefer based on their similarity to other users
12 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Recommendation Systems (RSs) pioneered the web with the aim of incorporating social informationand at the same time delivering meaningful suggestions to the end user While the research field ofRSs has been skyrocketed in diverse domains there is a gradual interest of the adoption of RSs inthe 5G ecosystems through their pinpointing in the network management applications [25] [26]
In this context many algorithms have been used in measuring user similarity or item similarityin recommendation systems For example singular-value decomposition (SVD) approach At thesame time Collaborative filtering is based on the assumption that people who agreed in the pastwill agree in the future and that they will like similar kinds of items as they liked in the past
bull Collaborative Filtering A key advantage of the collaborative filtering approach is thatit does not rely on machine analyzable content and therefore it is capable of accuratelyrecommending complex items such as tests without requiring an ldquounderstandingrdquo of the itemitself
bull Singular-Value Decomposition (SVD) In linear algebra the singular-value decomposi-tion (SVD) is a factorization of a real or a complex matrix It is the generalization of theeigendecomposition of a positive semidefinite normal matrix (for example a symmetric matrixwith positive eigenvalues) to any m x n matrix via an extension of the polar decompositionIt has many useful applications in signal processing and statistics
In our case and based on the above mentioned criteria tng-vnv-dsm is using SVD for measuringthe useritem similarity
3131 Release 50
Release 50 is the first release in which 5GTANGOrsquos SDK ships tng-vnv-dsm As a result there is nochange log given as all developments can be considered as new Itrsquos main supported functionalitiesare
bull Retrieve Componentrsquos health
bull Retrieve the items (testing tags) the recommendation engine is trained for
bull Retrieve the users list the recommendation engine is trained for
bull Add a new user-item pair based on the uploaded package to the catalogues
bull Get the top N recommendations for a user
bull Delete a user among with hisher associate activity
3132 Architecture
Scope
During their lifetime network services undergo many changes and such changes can often affectadversely the whole system Software engineers need to test the NSrsquos overall functionality beforedeploying a new product release to avoid undesirable changes or unexpected bugs Software test-ing is one of the common ways of evaluating system quality in a sequence of releases Softwareengineers validate the software system to ensure that no new faults have been introduced by newchanges However and since NSs are frequently evolving to the upcoming world of 5G and NFVtheir size and complexity are growing rapidly making the cost of testing too expensive Many test-ing and maintenance approaches have been proposed to reduce that cost including test selection
5GTANGO Public 13
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 33 Workflow between the portal and the recommender
and test prioritization Recommendation systems were used to mitigate the decision-making effortby providing users with a list of relevant items based on a userrsquos preference or item attributes Forexample companies producing daily-life applications such as Netflix Amazon and many socialnetworking applications are adopting recommendation systems to provide more personalized ser-vices to attract more users Recently in software engineering areas recommendation systems havebeen used to improve different software engineering tasks
Work Flow
The workflow described in Fig 33 shows how a user will be provided with test recommendationsbased on hisher profile activity while also based on similar userrsquos previous activities
REST - API httprec system ip address4010tng-vnv-dsmapiv1
Userrsquos Recommendations Example
An incremental data loading method have been used to design and implement the recommendationengine Incremental data loading is used when there is no previous data for training the systemThus it leads to reduced cost complexity since there is no need for perpetual training A user loggedinto the Portal can be provided with test recommendations based on hisher previous activity (basedon the previous uploaded packages or on the previous set of tests executed) Thus the response canbe used to recommend to the user possible matches on tests that he or she may like An exampleof the provided response is shown below
json
user tango_user
rec_tests [
testing_tag_X
testing_tag_Y
]
14 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3133 Installation
Installing tng-vnv-dsm itself is simple and it can be used as a standalone micro-service as describedin its GitHub repository [2] However the 5GTANGO VampV platform needs to be installed andconfigured in order to make an effective use of the tool To carry out this setup we provide adetailed online documentation in [38]
3134 Usage
An API has been designed and implemented for interacting with the recommendation engine whichcan be found here [1]
314 Descriptor generator and project management
The 5GTANGO project management has been an established SDK tool since the initial releaseand is an essential part of the development workflow It enables developers to set up their localworkspace and start creating NFV projects that contain VNF and network service descriptorslicense files logos or any other relevant artifacts using the toolrsquos simple CLI which is aligned tothe well-known git CLI [11] The project management tool keeps track of all involved artifacts inthe project using a project manifest The manifest also contains metadata about the project andthe individual files This metadata enables handling and distinguishing artifacts for 5GTANGOand other platforms such as OSM as shown at NFV-SDNrsquo18 [40]
In the last development cycle the project management tool was combined and integrated withthe descriptor generation tool for a more seamless workflow When creating a new NFV projectdevelopers can now directly generate suitable descriptors in a single step using the established CLIThese generated descriptors can then be adjusted to the specific use case without having to write lotsof repetitive boiler-plate code from scratch In addition the tool can be deployed as microserviceexposing a REST API which facilitates the integration with the SDK portal (Sec 312) andother web-based tools Release 50 also provides an improved extensible architecture with modulargeneration plugins that enable the descriptor generation for different platforms
3141 Release 50
bull Integration with project management tool Generate new descriptors automatically whencreating a new project (if desired)
bull Implemented REST API for microservice usage (Docker container)
bull Extended REST API allows integration in SDK portal which replaces the previous stand-alone GUI for a seamless workflow (Sec 3122)
bull Improved architectural design of the descriptor generator with different modular plugins forimplementing generation functionality for different platforms
3142 Architecture
The descriptor generation tool is now integrated with the project management tool Within the toolthe two functionalities for creating and editing projects and descriptor generation are still separatedinto two modules that interact with each other This separation ensures easier maintainability
In release 50 the descriptor generation module was improved to be more extensible and modular-ized Fig 34 shows how the generation of descriptors is handled by several independent and mod-ular generation plugins that correspond to the different platforms of interest such as 5GTANGO
5GTANGO Public 15
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 34 Improved extensible architecture with modular generation plugins for different plat-forms (eg 5GTANGO OSM or ONAP)
16 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
OSM [27] or ONAP [32] Within each plugin there are default descriptors for VNFDs and NSDswhich contain typical boilerplate information that is similar in most descriptors Furthermoreeach plugin contains a module with the generation logic which replicates and adapts the defaultdescriptors based on given high-level information for descriptor generation
The clean separation of the individual plugins allows easy maintainability and updates for dif-ferent platforms as well as adding support for new platforms Currently 5GTANGO and OSM aresupported and support for ONAP is envisioned
3143 Installation
The installation and setup of the tool is simple and an up-to-date description can be found online[38] as further described in Sec 71
3144 Usage
The tng-sdk-project tool can be used either via its simple CLI or using its exposed REST APIwhen deployed as microservice Similar to the installation a detailed up-to-date usage descriptioncan be found online [38] In the following general usage information and small examples areprovided to illustrate the toolrsquos CLI and microservice functionalities
The following listing shows the synopsis of all CLI arguments supported by the tng-sdk-projecttool
$ tng-project -h
usage tng-project [-h] [-v] [-p PROJECT] [-w WORKSPACE] [--empty] [--add ADD]
[-t TYPE] [--remove REMOVE] [--status] [--translate]
[-o OUT_PATH] [--tango] [--osm] [--author AUTHOR]
[--vendor VENDOR] [--name NAME] [--description DESCRIPTION]
[--vnfs VNFS]
[--image_names [IMAGE_NAMES [IMAGE_NAMES ]]]
[--image_types [IMAGE_TYPES [IMAGE_TYPES ]]] [-s]
[--dump-swagger] [--address SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK project
optional arguments
-h --help show this help message and exit
-v --debug increases logging level to debug (default False)
-p PROJECT --project PROJECT
create a new project at the specified location
(default new-project)
-w WORKSPACE --workspace WORKSPACE
location of existing (or new) workspace If not
specified will assume rsquoCUsersStefantng-workspacersquo(default None)
--empty create an empty project (without sample files)
(default False)
--add ADD Add file to project (default None)
-t TYPE --type TYPE MIME type of added file (only with --add) (default
5GTANGO Public 17
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
None)
--remove REMOVE Remove file from project (default None)
--status Show project file paths (default False)
--translate Translate old SONATA project to new 5GTANGO project
(default False)
-o OUT_PATH set relative output path (default )
--tango only generate 5GTANGO descriptors (default False)
--osm only generate OSM descriptors (default False)
--author AUTHOR set a specific NSD and VNFD author (default 5GTANGO
Developer)
--vendor VENDOR set a specific NSD and VNFD vendor (default
eu5gtango)
--name NAME set a specific NSD name (default tango-nsd)
--description DESCRIPTION
set a specific NSD description (default Default
description)
--vnfs VNFS set a specific number of VNFs (default 1)
--image_names [IMAGE_NAMES [IMAGE_NAMES ]]
list of VNF image names (default ubuntu) (default )
--image_types [IMAGE_TYPES [IMAGE_TYPES ]]
list of VNF image types (default docker) (default )
-s --service Run tng-project in service mode with REST API
(default False)
--dump-swagger Dump Swagger JSON of REST API and exit (default
False)
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
(default 0000)
--port SERVICE_PORT TCP port of REST API when in service mode (default
5098)
Usage example showing how to first create a new project with generated descriptors and thenadding and removing further files
$ tng-project -p pathtoproject --author abc --vnfs 2 --image_names img1 img2 --image_types docker docker
$ tng-project -p pathtoproject --add file1
$ tng-project -p pathtoproject --add file1 --type textplain
$ tng-project -p pathtoproject --remove file1
$ tng-project -p pathtoproject --status
Microservice
The tng-sdk-project tool containing integrated descriptor generation and project managementcapabilities can be deployed as light-weight microservice running in a Docker container TheDocker image can either be built locally after obtaining the source code (see Sec 7) which may beuseful for some developers or simply be pulled from DockerHub [29]
After stating the tool in service mode (ie not using its CLI) it continuously runs exposing aREST API Fig 35 shows an excerpt of the APIrsquos specification using swagger This REST API
18 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 35 Overview of the tng-sdk-project REST API
5GTANGO Public 19
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
also supports the newly integrated descriptor generation functionalities as shown in the followingexample
create a new project
$ curl -X POST localhost5098apiv1projects
show all projects
$ curl -X GET localhost5098apiv1projects
new project with custom-generated descriptors
$ curl -X POST localhost5098apiv1projects -d author=alice -d vendor=eutango -d vnfs=3
you can specify image namestypes as white space-separated list
$ curl -X POST localhost5098apiv1projects -d vnfs=2 -d image_names=img1 img2
show details of the specified project
$ curl -X GET localhost5098apiv1projectsuuid delete the specified project
$ curl -X DELETE localhost5098apiv1projectsuuid
The extended REST API is the basis for the integration with the SDK portal as described inSec 3122
315 Packager
The packer (tng-sdk-package) is one of the key components of 5GTANGO Even if it is developedtogether with 5GTANGOrsquos package standard [16] as part of the SDK it is also used as part of theservice platform gatekeeper and the gatekeeper of the VampV Every package for example the servicepackages used to deploy the three pilots of 5GTANGO is touched twice by this component Firstduring development when the package is created Second during on-boarding when the packageneeds to be unpacked and the contained artifacts are uploaded to the catalogs
During the last release cycle several improvements features and fixes have been added totng-sdk-package as we describe in the following sections Two highlights are the integrationof 5GTANGOrsquos SDK validator to automatically validate all packedunpacked packages (SectionSec 3153) and the OSM-compatible storage backend that allows direct artifact upload to OSM(Sec 3153) which was demonstrated at IEEE NFV-SDNrsquo18 [40] and underlines the usefulness ofthis component outside the scope of the 5GTANGO ecosystem The package format in contrastturned out to be well designed and only minor compatibility updates and bug fixes were neededduring the last release cycle
3151 Release 50
Overview of of changes from the release notes
bull Integration tng-cat storage backend
bull Integration Auto validation using tng-sdk-validation
bull Integration Aligned all logging to 5GTANGO standard
bull Integration Multi-user support
bull Integration Multi-platform support (OSMONAP) for tng-cat
20 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
bull Integration Support OSM as storage backend
bull Integration Testing tags for integration with VampV
bull REST API Health check endpoint
bull REST API Expose detailed processing status
bull CLI Packagingunpackaging reports
bull CLI Unpackaging to local filesystem
bull CLI ndashquiet flag for integration with tng-sdk-benchmark
bull CLIREST Added autoversion feature to increase package versions automatically upon pack-aging
bull Fix Several dozens of minor fixes and improvements
3152 Installation
The installation of tng-sdk-package can be done together with the other 5GTANGO SDK toolson a developerrsquos laptop Alternatively the packager can be executed as a Docker container as part ofa microservice The installation procedures are described in 5GTANGOrsquos official quick guide whichcan be found online [38] We do not provide those installation procedures in this static documentsince they may be subject to change and thus better documented in a living online document
3153 Usage
CLI
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-package
release 50
$ tng-pkg -h
usage tng-pkg [-h] [-p PACKAGE] [-u UNPACKAGE] [-o OUTPUT]
[--format PKG_FORMAT] [-v] [--loglevel LOG_LEVEL] [--logjson]
[-q] [--ignore-checksums] [--skip-autoversion]
[--skip-validation] [-w WORKSPACE] [--offline] [--store-skip]
[--store-backend STORE_BACKEND] [-s] [--dump-swagger]
[--dump-swagger-path DUMP_SWAGGER_PATH]
[--address SERVICE_ADDRESS] [--port SERVICE_PORT]
5GTANGO SDK packager
optional arguments
-h --help show this help message and exit
-p PACKAGE --package PACKAGE
Create package from given project
-u UNPACKAGE --unpackage UNPACKAGE
Unpackage given package
-o OUTPUT --output OUTPUT
Path to outputs (optional)
5GTANGO Public 21
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--format PKG_FORMAT Package format [eu5gtango|euetsi|euetsiosm]
Default eu5gtango
-v --verbose Output debug messages
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-q --quiet Do not print packaging info
--ignore-checksums Do not validate artifact checksums
--skip-autoversion Auto increase package version field
--skip-validation Donrsquot call the validator during packunpack
-w WORKSPACE --workspace WORKSPACE
Location of existing workspace (see tng-project -h)
If not specified will assume rsquoUsersmanueltng-
workspacersquo
--offline Donrsquot resolve online resource like schemas for
validation
--store-skip Skip store step
--store-backend STORE_BACKEND
Storage backend to be used Default
TangoProjectFilesystemBackend
-s --service Run packager in service mode with REST API
--dump-swagger Dump Swagger JSON of REST API and exit Default False
--dump-swagger-path DUMP_SWAGGER_PATH
Path to dump Swagger JSON using --dump-swagger
Default docrest_api_modeljson
--address SERVICE_ADDRESS
Listen address of REST API when in service mode
Default 0000
--port SERVICE_PORT TCP port of REST API when in service mode Default
5099
Usage example to package a 5GTANGO SDK project
$ tng-pkg -p misc5gtango_ns_project_example1
======================================================
P A C K A G I N G R E P O R T
======================================================
Packaged misc5gtango_ns_project_example1
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output eu5gtango5gtango-project-sample01tgo
Error None
Result Success
======================================================
Usage example to unpack a 5GTANGO package to the local file system
$ tng-pkg -u misc5gtango-ns-package-exampletgo
22 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
===================================================
U N P A C K A G I N G R E P O R T
===================================================
Unpackaged misc5gtango-ns-package-exampletgo
Project eu5gtango5gtango-project-sample01
Artifacts 2
Output 5gtango-ns-package-example
Error None
Result Success
===================================================
Service API
The REST API used to integrate tng-sdk-package with other components of the service platformand VampV has been largely stable in the last release cycle Only minor changes like optionalparameter fields in the package upload endpoints have been required An example for this isthe additional username parameter which was added to all endpoints to add multi-user supportto this component Fig 36 shows the latest version of the automatically generated interactiveAPI documentation which is available online [17] This online resource contains the full APIspecification including all data models
One model that was updated for release 5 is the data structure that is sent as part of thecallback when tng-sdk-package finishes the unpackaging procedure This data structure nowlooks as follows
event_nameonPackageChangeEvent
package_id24c616cf-fe01-4c08-ae44-45d43ae67576
package_locationhttpcatcataloguesapiv2tgo-packagesuuid
package_metadata []
package_process_uuidd5cea225-033f-4fc6-816f-4a642461086a
package_process_status success
It now also contains an URI that directly points to the storage location in the catalog whichmakes this callback message more useful for other components that call the packager They donot need to query the catalog in order to find recently uploaded package contents but can directlyaccess them following the given link This reduces the interactions inside the 5GTANGO platformand improves its performance
Integration of Validator
One of the key features of release 50 is the integration of the tng-sdk-validate tool and thetng-sdk-package tool The rationale behind this feature is that developers using the 5GTANGOSDK as well as the 5GTANGO VampV and SP have a high interest in ensuring that the manipulatedor used descriptors have the correct format Thus it makes sense to always validate them whenthey enterleave a system or an environment Since 5GTANGO uses packages to exchange artifactsbetween platforms and environments and thus it was an obvious choice to always run the validationwhen a package is packedunpacked
To integrate both tools the packager directly imports the Python modules of the validator andcalls them through their Python APIs The reason for this design in contrast to an additional
5GTANGO Public 23
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 36 Latest version of automatically generated OpenAPI documentation of REST API end-points
24 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 37 PackagerValidator call graph for packaging example
Figure 38 Usage of tng-sdk-package as part of OSM to make OSM compatible to the advanced5GTANGO package format
REST API between the tools is that the packager is installed locally on a developerrsquos laptop wherea REST-based approach would have been problematic Fig 37 shows the final integration and callgraph of both tools for an example packaging procedure
Using OSM as storage backend
As described in D41 [20] the 5GTANGO package format [16] comes with some advanced fea-tures eg for testing Those features are not supported by CSAR ETSI or OSM packages yet[28 31] To work around those shortcomings of other platforms we added the concept of storagebackends to our packager tool Those storage backends can either be used to connect the pack-ager with different catalog solutions but they can also be used to extend existing platforms withsupport for the advanced 5GTANGO package format Fig 38 shows such a scenario in which weadded tng-sdk-package to an OSM platform to build an OSM prototype that accepts 5GTANGOpackages
To do so we added a new storage backend to tng-sdk-package which can optionally be activatedand is able to on-board unpackaged artifacts to a given OSM installation This is done by packaging
5GTANGO Public 25
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
each individual artifact into an original OSM package and upload these intermediate packagesThus none of the existing OSM APIs need to be changed and we have a fully transparent approachto extend existing NFV MANO solutions We demonstrated this at 2018 IEEE NFV-SDN [40]
316 Validator
The validator (tng-sdk-validation) is one of the components of the 5GTANGO project Itsmain goal is to check the syntax integrity and topology of the different descriptors related to anNFV project namely services functions tests slices SLAs and policies In addition this toolcan include specific checks which are called custom rules validations For example the servicepackages of each of the three pilots can be analysed to confirm if the descriptors include all theconnections required for that service so that everything will be expected to communicate properlyonce deployed
For release 50 several improvements and bug fixing have been made This tool has been inte-grated with tng-sdk-package so that every time a package is packed or unpacked a new validationprocess starts This was very helpful to deeply test the tool and proceed with corresponding fixes
3161 Release 50
Overview of changes from the release notes
bull Support for updated descriptor schemes
bull Support for CNF descriptors
bull Support for Kubernetes descriptors
bull Support for SLA policy and slice descriptors
bull Support for test descriptors
bull Support port validation for CDUs in CNFs
bull Implemented automatic and periodic storage of descriptor schemas
bull Implemented advanced custom rule validation and updated rule descriptor
bull Logs improvement
bull Unit tests update
bull Bug fixes
3162 Installation
The installation of tng-sdk-validation can be done together with the other 5GTANGO SDKtools on a developerrsquos laptop Besides the validator can also be installed as a Docker containeras part of a microservice The installation procedures are described in 5GTANGOrsquos official quickguide which can be found online [38] We do not provide those installation procedures in thisstatic document since they may be subject to change and thus better documented in a living onlinedocument
26 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3163 Usage
The validator can either be used as command line tool (CLI mode) or deployed as a micro servicewhich offers a REST API
CLI
Running the validator locally from the command line we obtain a list of all the possible parametersthat can be used in release 50
CLI input arguments [rsquo-hrsquo]
usage tng-sdk-validate [-h] [-w WORKSPACE_PATH]
(--project PROJECT_PATH | --slice NST | --policy RPD |
--sla SLA | --service NSD | --function VNFD |
--test TSTD | --api)
[--dpath DPATH] [--dext DEXT] [--syntax] [--integrity]
[--topology] [--custom] [--cfile CFILE] [--debug]
[--mode statelesslocal] [--host SERVICE_ADDRESS]
[--port SERVICE_PORT]
5GTANGO SDK validator
optional arguments
-h --help show this help message and exit
-w WORKSPACE_PATH --workspace WORKSPACE_PATH
Specify the directory of the SDK workspace for
validating the descriptors of SDK project
--project PROJECT_PATH
Validate the service descriptors of the specified SDK
project
--slice NSTD Validate the specified netwok slice template
descriptor
--policy RPD Validate the specified runtime policy descriptor
--sla SLAD Validate the specified SLA descriptor
--service NSD Validate the specified service descriptor The
directory of descriptors referenced in the service
descriptor should be specified using the argument rsquo--
pathrsquo
--function VNFD Validate the specified function descriptor If a
directory is specified it will search for descriptor
files with extension defined in rsquo--dextrsquo
--test TSTD validate the specified test descriptor
--api Run validator in service mode with REST API
--dpath DPATH Specify a directory to search for descriptors
Particularly useful when using the rsquo--servicersquo
argument
--dext DEXT Specify the extension of descriptor files
Particularly useful when using the rsquo--functionrsquo
argument
--syntax -s Perform a syntax validation
5GTANGO Public 27
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
--integrity -i Perform an integrity validation
--topology -t Perform a network topology validation
--custom -c Perform a network custom rules validation
--cfile CFILE Specify the file with the custom rules to validate
--debug Sets verbosity level to debug
--mode statelesslocalSpecify the mode of operation rsquostatelessrsquo mode will
run as a stateless service only rsquolocalrsquo mode will run
as a service and will also provide automatic
monitoring and validation of local SDK projects
services etc that are configured in the developer
workspace
--host SERVICE_ADDRESS
Bind address for this service
--port SERVICE_PORT Bind port number
Some examples of usage
- Validation of project descriptors in a particular workspace
tng-sdk-validate --project pathtoproject --workspace pathtoworkspace
- Validation of project descriptors in the default workspace
($ HOMEtng-workspace)
tng-sdk-validate --project pathtoproject
- Validation of service descriptors
tng-sdk-validate --service pathtoexample_nsdyml --dpath pathtofunction_folder --dext yml
- Validation of all function (VNFCNF) descriptors in a folder
tng-sdk-validate --function pathtofunction_folder
tng-sdk-validate --function pathtofunction_folder --dext yml
- Validation of individual function (VNFCNF) descriptor
tng-sdk-validate --function pathtoexample_functionyml
tng-sdk-validate --function pathtoexample_functionyml --dext yml
- Validation of individual test (TSTD) descriptor
tng-sdk-validate --test pathtoexample_testyml
tng-sdk-validate --test pathtoexample_testyml --dext yml
- Validation of individual network slice template (NST) descriptor
tng-sdk-validate --slice pathtoexample_sliceyml
tng-sdk-validate --slice pathtoexample_sliceyml --dext yml
- Validation of individual sla (SLA) descriptor
tng-sdk-validate --sla pathtoexample_slayml
tng-sdk-validate --sla pathtoexample_slayml --dext yml
28 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 39 Latest version of automatically generated OpenAPI documentation of tng-sdk-validationrsquos REST API endpoints
- Validation of individual runtime policy (RPD) descriptor
tng-sdk-validate --policy pathtoexample_policyyml
tng-sdk-validate --policy pathtoexample_policyyml --dext yml
REST API
The REST API has only been updated to support the new type of validations included in this lastrelease The latest version of the automatically generated interactive API documentation (shownin fig 39) is available online [18]
317 Testing framework
One of the most promising benefits of NFV is DevOps automation However one of the biggestchallenges in DevOps environments is testing of network services against different execution plat-forms so that service operators can be sure that network functions and services behave as expectedimmediately after they are deployed and put into production
5GTANGO introduces a new workflow for testing network services from local environments
5GTANGO Public 29
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to different service platforms The key 5GTANGO component for testing is the Validation andVerification platform ([22]) In addition 5GTANGO SDK provides tng-sdk-test framework andthe Emulator to support local testing Below we give an introduction to tng-sdk-test and describehow local tests can be reused on the VampV platform
3171 Release 50
Release 50 is the first release in which tng-sdk-test is shipped as part of 5GTANGOrsquos SDK Asa result no change log is given since all developments can be considered as new
3172 Architecture
tng-sdk-test is a Python-based framework for automated functional testing of network functionsand services It can be used to create tests run them on different platforms and retrieve andanalyze test results The Python language was chosen for its simplicity and availability of manythird-party libraries which can be used for complex test results analysis Limited performance ofPython compared to compiled languages is not an issue since the main focus of the frameworkis functional testing Moreover Python code can call methods from CC++ or Java libraries forperformance-critical tasks
The framework does not depend on any Python testing libraries and can be used with any ofthem ie UnitTest from the Python standard library or more powerful PyTest ([6]) In a test codethe developer selects which infrastructure to use which network functions or services to deployhow to exercise and verify the test results
The testing framework has a separate module for interacting with virtual infrastructure managersCurrently it supports the 5GTANGO Emulator for local testing and the VampV platform to executetests on real environments The framework can be easily extended to support any other VIMBelow we describe how the framework can be used for local testing and the requirements forseamless workflow from local testing to the VampV
Local testing
The 5GTANGO Emulator ([Sec 3110]) is a light-weight emulation platform which can be deployedeven on low-performance machines such as laptops With the help of the Emulator the testingframework can be used by NS developers to run functional tests locally Moreover it can be usedfor automated testing in CICD environments with no need to access complex infrastructure
First the test code should define which instances will be used When running tests locally thereis no difference between network functions under test and additional instances that are used fortesting purposes Network functions can be added in multiple ways from a package an imageor from source The first option should be used to test a complete network service In this casethe framework instantiates appropriate network functions and sets network links according to thedefinition in the descriptors When network functions are not added using a package network linksshould be added manually using special methods in a test code The framework can be instructedto add a traffic sniffer on some links so that this traffic can be retrieved and analyzed later to verifytest execution results Adding network functions from source is particularly useful for automatedtesting since repositories usually contain source code of software In this case the framework willbuild a new image every time the test is executed
Once the system-under-test and all additional VNFs are deployed arbitrary commands or scriptscan be executed on these instances to exercise the system Finally the output of the executedcommands files records in journals of instances or sniffed traffic from added links can be retrieved
30 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
to verify the test results Any of the available third-party Python libraries can be used to analyzethe test results
Running tests on the VampV platform
In order to support seamless workflow the testing framework provides several tools for migratingtests to the VampV platform Firstly there are some restrictions for tests that can be executed onthe VampV platform
bull The test code has to be agnostic to the platform it is running on
The test code can assign a concrete VIM to be used for the test or can let the testing frameworkdecide which VIM to use based on the environment variables In order to use the same code on allplatforms the last option must be used
bull The VampV platform distinguishes services under test and additional test functions which arecalled probes
Network services are uploaded on the VampV platform as service packages and are then deployedon real infrastructure In contrast probes come as container images and are deployed on the VampVplatform itself This means the test code must add at least one network service using a packagecan not manipulate with network configuration execute commands and read data from instancesof network services When running tests locally the testing framework can check if the test satisfiesthese requirements and fail it if the requirements are not met
Secondly the testing framework generates a Docker image with the test The image containsa Python interpreter the testing framework and the test codes Moreover the testing frameworkanalyses the test code makes a list of dependencies and installs them inside the image
Thirdly the framework generates a test descriptor A test descriptor contains test metadatatesting tags used to map a test with an appropriate network service define probes and theirparameters specifies how to run these probes and where to store and how to verify the outputThe testing framework parses the content of the network service packages to retrieve informationabout network functions external connection points of network services and testing tags Thegenerated test descriptor defines a single probe which is the Docker image from the previous stepThe parameters of the probe contain placeholders for external interfaces of the network serviceWhen the test is running on the VampV platform it deploys the network service retrieves informationabout running instances from the service platform and substitutes placeholders in the test descriptorwith this information
Once the test descriptor is generated it can be packaged with tng-sdk-package (see Sec 315)and uploaded on the VampV platform When the VampV starts the probe it mounts the Docker Enginesocket and executable as volumes which gives access to the Docker Engine The testing frameworkinside the probe uses it to run additional test VNFs defined in the test code
Finally the probe generates two files with the logs and with the test exit-code The verificationpart of the test descriptor instructs the VampV platform to verify that the exit-code is zero In caseof a failure the logs can be analyzed to find the errors in the network service
3173 Installation
The framework can be installed using the following commands
git clone httpsgithubcomsonata-nfvtng-sdk-test
cd tng-sdk-test
python setuppy install
5GTANGO Public 31
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
or using pip
pip install git+httpsgithubcomsonata-nfvtng-sdk-test
In order to run tests locally the Emulator has to be installed separately See [Sec 3110] for theEmulator installation instructions
3174 Usage
The documentation of the framework is available at [8] In order to use the framework it should beinstalled on the system and imported in a test code Some examples of tests using the frameworkare available on [9] Here is a list of available methods
vim = Vnv()
vim = Emulator(vnv_checker=False)
vim = vim_from_env()
vimadd_instances_from_package(path)
vimadd_instance_from_image(name image interfaces=None docker_command=None)
vimadd_instance_from_source(name path interfaces=None docker_command=None
docker_build_args=None)
vimadd_link(source_vnf source_interface dest_vnf dest_interface
sniff=False)
vimmy_vnfexecute(command)
vimmy_vnfexecute(script)
vimmy_vnfget_file(path)
vimmy_vnfget_journal(service filter=None)
vimget_traffic(source_vnf source_interface dest_vnf dest_interface)
create_vnv_test(source package descriptor=None probe=None)
318 Development and testing of specific manager functionality
The tng-sdk-sm tool was first introduced in Release 40 of SONATA Its was added to the SDK toaid network service and function developers with the creation and testing of their Service SpecificManagers (SSM) and Function Specific Managers (FSM) Its goal is to provide developers with aframework that allows them to test their specific managers in a local setup overcoming the slowand cumbersome process of having to interface with external (operator) components to obtaininformation on how the specific managers are functioning Release 40 mainly focused on thecreation of specific managers and the testing of Function Specific Managers With Release 50 oftng-sdk-sm Service Specific Managers are now also supported
3181 Release 50
Overview of changes since last release
bull Support for the testing of Service Specific Managers
bull Simplification of the Specific Manager model
32 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3182 Architecture
Scope
5GTANGO allows developers to add Service and Function Specific Managers to the descriptorsof network services and functions This is a distinctive feature of SONATA and 5GTANGO asit enables customisation of otherwise rigid orchestration and configuration capabilities availablein other MANO platforms These specific managers are processes which interact with the ServicePlatform using a well-defined API over the SP pubsub bus [19] and incorporate service or functionspecific orchestration behaviour Service Specific Managers customise orchestration behaviour forNS life cycle events while Function Specific Managers customise VNF orchestration events Thedeveloper adds them to the relevant descriptor together with detailing which workflow they cus-tomise or extend When the Service Platform needs to execute a network service life cycle eventit will first check whether one or more SSMs are associated with this service and workflow Ifthat is the case the Service Platform will execute this SSM ie process instead of executing thegeneric workflow A similar behaviour can be expected for network function life cycle events andFSMs SM functionality enables NS- or VNF-specific placement START- and STOP behaviourconfiguration monitoring scaling migration and fail-over
Over the course of SONATA it was proven that developing and testing specific managers wasa slow process and error-prone For this reason D41 introduced tng-sdk-sm into the SDK atool that generates SSM and FSM templates and allows the developer to test their functionalitylocally after adjusting them speeding up their development significantly and overcoming the cum-bersome process of interacting with operator logging components to obtain debug information onthe functioning of these specific managers
Testing Service Specific Managers
With Release 50 tng-sdk-sm now supports local testing of SSMs From the toolrsquos perspectivethis is more challenging than FSM testing FSMs are used by the SP MANO Framework tointeract with the VNF VMs and containers making them the last shackle in the chain of a MANOFramework workflow Therefore they can be tested by mocking MANO Framework inputs andevaluate whether the resulting interaction with a VM or container was executed as expected SSMshowever are part of the MANO Frameworkrsquos core operations They can trigger new workflowsthey can alter workflows they can provide custom payloads to FSMs that will result in differentVM or container configurations etc
Because of this wide variety of SSM outcome mocking MANO Framework interaction to testSSMs quickly becomes very complex It would require anticipating all possible outputs of an SSMwith appropriate follow-up MANO Framework requests to continue the workflow in which the SSMis being tested Since this logic is already present in the actual MANO Framework it makes moresense to use the real MANO Framework instead of developing a mock with a lot of duplicate codeTherefore we refactored the SONATA MANO Framework so it can operate stand-alone withoutneeding most of the other SP components in a local environment The reason we deploy onlythe MANO Framework and not the entire SP is to have a quicker and more lightweight setup tofurther decrease testing times and reduce any stress on local resources These are the componentsthat are part of the local MANO Framework setup
bull RabbitMQ Message Broker
bull MongoDB
bull MANO Framework
5GTANGO Public 33
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
ndash Service Lifecycle Manager
ndash Function Lifecycle Manager
ndash Plugin Manager
ndash Placement Plugin
ndash Specific Manager Registry
bull Repositories
bull Emulator Wrapper
For the testing purposes we approach the MANO Framework as a black box We use its north-bound API (not through the Gatekeeper as is the case in the SP but through the message broker)to request it to execute certain workflows If these workflows include the use of an SSM the MANOFramework automatically deploys and uses them To test the functionality of the SSM we waituntil the MANO Framework is finished with the requested workflow and then evaluate the resultof this workflow For this evaluation we can access the SSM logs the MANO logs and the MANOoutcome report
Since almost all MANO Framework workflows require interaction with the infrastructure thatis being orchestrated we need to make sure that it receives valid responses when making requeststowards to infrastructure Since SSMs and FSMs might require to interact with VNF VMs or con-tainers directly mocking the infrastructure is not an option Indeed with a mocked infrastructurea specific managers will never be able to set up an SSH connection with a VM resulting in specificmanagers that canrsquot be tested As we donrsquot want to expect the developers to have actual infras-tructure available we make use of the SDKrsquos Emulator tool which allows any developer to emulateboth compute and networking resources in their local environment In order for the MANO Frame-work to be able to orchestrate on these emulated resources we developed an Emulator wrapperthat translates MANO Framework infrastructure requests into calls for the Emulator REST API
With the lightweight setup of the SONATA MANO Framework the Emulator wrapper and theEmulator we can now quickly test SSMs in a local environment Fig 310 shows the entire localsetup created by tng-sdk-sm in order to test SSMs
Simplification of the Specific Manager Model
As feedback on earlier releases the Specific Manager Model was deemed cumbersome Numerousfields (eg sm type sm name sm version sm id ) had to be present in both the specific managercode registration message and container name to obtain a successful on-boarding and attachmentof the specific manager within the SP To improve this a simplification of the model was proposedtogether with an associated refactoring of the MANO Framework From Release 50 onwards thereare no longer limitations on the container name of a specific manager and the only required fieldsin the registration message are
selfsm_id = ltidgt
selfsm_version = ltversiongt
3183 Installation
tng-sdk-sm tool is a golang application How to install it is described in the Readme of theassociated GitHub repository [13]
34 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 310 tng-sdk-sm local setup for SSM testing
3184 Usage
tng-sdk-sm can be used from the CLI as follows
usage tng-sm [--version] [--help]
These are the subcommands for lsquotng-smlsquo
new Create a new specific manager
delete Delete an existing specific manager
execute Execute an event of a specific manager
generate Generate artefacts to be used when executing specific managers
usage tng-sm new ltspecific manager namegt
--path Path where new specific manager should be stored
--type Type of specific manager to be created ssm or fsm
usage tng-sm delete ltspecific manager namegt
--path Path where specific manager can be found
usage tng-sm execute ltspecific manager namegt
--path Path where specific manager can be found
--event Event that needs to be executed
--payload Payload for the execution
5GTANGO Public 35
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
usage tng-sm generate ltname output filegt
--type Type of payload to be generated vnfr or nsr
--descriptor File that serves as input for generation should be a vnfd
or nsd
319 State management support
Most network functions need to store a certain amount of state The type and the extent of statethat a particular network function keeps depends on the purpose of a function as well as on its im-plementation Typically the state is kept locally inside the respective VNF For certain use casesthough the state needs to be transferred from one location to another As described in deliverableD22 [14] such use cases include in particular migration fail-over upgrade and scale-outin Fur-thermore as detailed in deliverable D41 [20] state management requires support from the MANOsystem for properly orchestrating the process for migration upgrade or scaling Although the typeof state is service-specific the processes for managing state can still be generalized and integratedwith most if not all services Such support includes run-time functions as well as tools whichfacilitate the development and testing of state-aware services Together they form an integral partof the NFV DevOps cycle
3191 Release 50
Release 50 is the first release in which state management functions are shipped as part of 5GTANGOAs a result no change log is given since all developments can be considered as new
3192 Patterns for state management
State management as required by the above mentioned use cases typically involves a small numberof basic activities These basic activities are exporting state importing state transferring stateand storing state Scaling processes might also include activities related to splitting and mergingstate upgrading might require the translation of state data
Based on these basic activities a number of state management patterns can be described Thesepatterns are (i) direct state transfer (ii) state replication and (iii) state proxying as shown inFig 311 and described in the following
bull Direct state transfer refers to transferring state directly from one VNF to another (Fig 311a)In this process the source VNF gathers all its internal state and puts it into some formatthat allows the complete state to be moved around In the direct state transfer case thestate data is then sent to the destination VNF eg via a direct TCP connection Thedestination VNF accepts that state data and translates it into its internal data structuresThe destination VNF can now seamlessly take over the work of the source VNF Related tothe basic activities introduced before this process executes the following activities exportstate transferring state and import state
bull State replication is similar to direct state transfer as state is also directly sent from thesource VNF to one or more destination VNFs (Fig 311b) But while direct state transfer isa one-shot activity transferring state upon receipt of some explicit trigger state replicationis a continuous activity where state updates are constantly sent from the source to thedestination The basic activities of exporting transferring and importing state are the samefor both patterns but the data transferred is different it consists of the whole state for direct
36 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 311 State management patterns
state transfer but only state updates (ie differences to previous state) in the case of statereplication
bull The pattern of state proxying does not transfer state directly between source and destinationVNFs (Fig 311c) It rather has a third component (ldquoproxyrdquo) in between the source andthe destination which temporarily stores state The source VNF sends its state to the proxywhich stores the state It then later on hands that state out to the destination VNF In termsof basic activities state proxying executes them in the following order export state (at thesource VNF) transfer state (to the proxy) store state (at the proxy) transfer state (to thedestination VNF) import state (at the destination VNF)
Besides these explicit state handling patterns the stateless function pattern well-known fromWeb and cloud applications can be used In this pattern VNFs do not keep internal state at allbut rather use some external service to manage it typically a data base (including SQL no-SQLin-memory key-value store etc) This pattern can be seen as a combination of the state proxyingand the replication patterns as data is kept externally but state updates are continuously sent tothe data base This pattern is not explored here as it is usually realized on the application layerwithout support of the MANO system
When looking at the basic activities it can be noted that activities for exporting and importingstate are specific to the function and implementation of each VNF Although these activities canbe supported by the programming language or framework (eg serialization in the Java language)they are not an NFV MANO-related activity and are therefore out of scope of the 5GTANGOproject
The activities for transferring and storing state can be generalized and used by many VNFsIn particular as they are part of the management and orchestration processes there should beappropriate support by the MANO system Such support has been investigated designed andimplemented prototypically by the 5GTANGO project
5GTANGO Public 37
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
3193 State transfer and storage support
In order to support state storage in a MANO environment a common storage space should beoffered by the system In particular the storage space should be managed by the MANO systemnot each individual network service or VNF In that way the lifecycle of the storage is independentof the lifecycle of the service This arrangement cleanly separates the responsibilities and tasksbetween the service developer and the service operator The service operator gains greater controlof the services and their environments as she can provision and manage the storage as fits herpolicies and priorities best Furthermore development of network services becomes easier andfaster as the application developer does not need to deal with acquiring and managing the storagespace himself As an additional benefit resource usage will drop if multiple services use the samestorage space instead of separate instances per service
Access to the storage space can be provided in many ways and needs to be in line with thecapabilities and requirements of the deployed services and VNFs Most existing software usedas VNFs already has some support for state transfer eg in the form of a replication scheme Ifsuch transfer mechanism already exists it is less invasive and usually less costly to use that existingscheme and not touch the code for adding a new one In these cases it makes sense to offer a storage(and transfer) interface that is matching the one already used by the VNF software eg the NFSprotocol
The choice of storage backend is largely independent of the access method used Data can bestored on the file system local to the MANO system or in cloud storage It can be in a key-valuestore or in an SQL data base In most cases multiple options will be available depending on thepreferences of the operator and the deployed services
In order to use a provided storage space a reference to it (eg IP address URL) must beprovided to VNFs How this happens is implementation specific but can generally be realizedvia adaptor components which on the one side receive information about storage spaces from theMANO system and on the other pass this information on to the VNFs in their specific fashionsA similar scheme of information transfer is needed for the direct state transfer pattern in order tomake the VNFs aware of their peer functions
3194 State management process support
Use of storage spaces as well as the orchestration of state management processes needs to be adaptedto the specific network services and VNFs The SONATA platform already provides such a service-specific adaptation mechanism via service-specific and function-specific managers (SSMsFSMs)These managers can be extended to also provide state management capabilities specific to theirmanaged entities
The managers need to map generic lifecycle events and associated information to the specificsof their managed services and functions For state management one such mapping is neededduring the configuration phase in which information about the location of a state storage spaceis communicated to the respective VNFs For example a VNF storing state in a key-value storeneeds to get information about where this store is located eg in the form of an IP address anda TCP port number Similarly a service storing state data as a file needs to get the location ofthe network file system (NFS) server the directory name and potentially access credentials Suchdata can be transferred to the VNFs in a specific way eg via Ansible or SSH The SSMsFSMscollaborate with the rest of the MANO system from which they retrieve details about availablestorage mechanisms and locations and pass the appropriate one on to the VNFs
The other lifecycle events which need to be mapped onto service-specific activities are the mi-gration upgrading and scaling processes In order to support these activities a number of lifecycle
38 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 312 Lifecycle process migration
events have been introduced which are invoked during the execution of the respective lifecycle pro-cesses These lifecycle events are export state and import state They are executed at appropriatetimes during the respective lifecycle processes As an example Fig 312 shows the componentsand lifecycle events involved in the migration process The lifecycle manager (LCM) orchestratesa migration request by first asking VNF1 via the associated function specific manager (FSM1) toexport its state The state is shipped to the SSM and is stored and transformed there if neededThen the state is passed on to the destination VNF2 via its associated FSM2
The triggers for these processes are not predefined and depend on the services the environmentand the preference of the service operator One possibility is to trigger events based on SLAs andassociated policies
3195 Tool support for state management
Incorporating the state management procedures from the MANO system takes some effort fromnetwork service and function developers The SDK provides tooling to help with that process Sincedeveloping and testing state management SSMs and FSMs can be time consuming and cumbersomethe tng-sdk-sm tool should be used to ease that process It can be used to create templates forempty specific managers and to test them once they are configured with the appropriate statemanagement logic The FSMs can be tested by spinning up their associated VNFs in a localenvironment adding some state to it and using the tng-sdk-sm tool to execute the FSM workflowthat extracts or injects the state The developer should then evaluate whether the entire processresulted in a correct state management For SSM testing tng-sdk-sm deploys the MANO systemlocally to go through entire workflows that require state management eg a VNF migration eventAt the end of the workflow the tng-sdk-sm tool evaluates whether the state management SSMoperated correctly More information on the tng-sdk-sm tool and how to use it can be found in
5GTANGO Public 39
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
sec 318 in this deliverable
3110 Emulator
5GTANGO actively maintains OSMrsquos NFV emulation platform vim-emu [33] which was initiallydeveloped as part of SONATA and then adopted by OSM as we described in [20] This makes theemulator one of the key projects in the collaboration between OSM and 5GTANGO A series ofimprovements have been introduced for release 50 which focus on both A better integration withOSM as well as full support for 5GTANGO workflows and artifacts The following sections describethese developments in more detail focusing on a novel component called 5GTANGO lightweight lifecycle manager (LLCM) and the integration between SONATA SP and vim-emu which is realizedthrough a new infrastructure wrapper
31101 Release 50
Overview of of changes from the release notes
bull Core Made codebase PEP8 conform
bull Core Improved unit test and made them compatible with pytest
bull Core Improved stability
bull 5GTANGO LLCM Created 5GTANGO LLCM based on SONATA dummygatekeeper fornative support of 5GTANGO packages
bull 5GTANGO LLCM Added support for CNFs (new descriptor models etc)
bull 5GTANGO LLCM Added support for multi-VDUCDU deployments
bull 5GTANGO LLCM Added support to deploy multiple service instances in the same emulatedenvironment
bull 5GTANGO LLCM Supporting configurable port mappings
bull 5GTANGO LLCM Fixed E-Line IP management support for E-LAN and E-Tree networks
bull OSM integration Improved Glance API and made it more robust
bull OSM integration Updated installation procedure
bull OSM integration Support for multiple network ports with same name
bull OSM integration Made fake-floating IPs route-able from OSM to support Juju
bull OSM integration Added API for full-stack testing with latest OSM
bull OSM integration Added chaining support based on Neutron API
bull OSM integration General integration and continuous integration testing with OSM rel FIVE
40 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31102 Architecture
5GTANGO LLCM
The 5GTANGO lightweight life cycle manager (LLCM) module extends the existing vim-emu
project and implements support to natively deploy 5GTANGO service packages on the emula-tor Where the general concepts and ideas of the LLCM are based on the ldquoDummygatekeeperrdquopreviously developed in the SONATA project the implementation of the LLCM was heavily up-dated
The LLCM as such implements a small orchestratormdashor MANO systemmdashwith limited function-ality focusing on testing the initial deployment of network services and the integration betweenVNFs To do so it implements two endpoints
1 Package upload This endpoint is compatible to the Gatekeeper API of the original SONATASP and allows a user to on-board a 5GTANGO service package to the emulator The LLCMunpacks this package and keeps all contained artifacts eg NS and VNF descriptors inits memory It does not use a catalogues system but volatile internal memory to have aclean environment whenever the emulator is re-started This makes it well suited for rapidprototyping tasks
2 Service instantiation This endpoint allows to instantiate the previously on-boarded servicesThis includes the instantiation of the VNFsCNFs as lightweight Docker containers andinterconnecting them with an overlay network deployed between the emulated PoPs Thisoverlay is established using VLANs to separate networks of different service instances fromeach other
There are two areas in the 5GTANGO project which make use of the LLCM First the tng-sdk-benchmark tool uses vim-emu as execution platform for the automated benchmarking experimentsThe benchmarking tool interacts with the LLCM to deploy the tested services as well as the probesused to stimulate them during benchmarking The second area of use is prototyping We usedvim-emu together with the 5GTANGO LLCM to prototype and demonstrate 5GTANGOrsquos smartmanufacturing pilot The entire pilot is deployable on the emulator and can thus be executed ona single laptop This was used to produce the results of the smart manufacturing paper publishedat EuCNC 2019 [39] and will be demonstrated at IEEE NetSoft 2019 [35]
Example The following brief example shows how the emulator is started using an exampletopology with two PoPs and how to on-board and instantiate an example network service with twoVNFs on it
Step 1 Start the emulator using a demo topology with two PoPs
call
~vim-emu$ sudo python examplestango_default_cli_topology_2_poppy
output (skipped)
containernetgt
Step 2 On-board the 5GTANGO network service package to the 5GTANGO LLCM
call
~vim-emu$ curl -i -X POST -F package=misceu5gtangoemulator-example-service01tgo http1270015000packages
5GTANGO Public 41
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
output
error null
service_uuid 8c7a9740-4a05-422a-8fa2-2a5fa34b16a0
sha1 9b64a73fe5889dd5ccefdf93742395d685ca7b25
size 3513
Step 3 Instantiate the on-boarded service
call
~vim-emu$ curl -X POST http1270015000instantiations -d
output
service_instance_uuid a0266390-7bcf-40ed-9d53-70fdc0dfc76e
Step 4 Check the resulting deployment
call
~vim-emu$ vim-emu compute list
output
+--------------+-------------+---------------+-------------------+
| Datacenter | Container | Image | Interface list |
+==============+=============+===============+===================+
| dc2 | vnf0vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
| dc1 | vnf1vdu01 | ubuntutrusty | mgmtinputoutput |
+--------------+-------------+---------------+-------------------+
Performance To give the reader an idea of the performance of the LLCM in terms of on-boarding and instantiation times we present some example results in Fig 313 The results showhow the two network services of the smart manufacturing pilot which contain between 3 and 4CNFsCDUs are on-boarded and instantiated on vim-emu The on-boarding time takes usuallyless than 09 seconds and both services can be instantiated in less than 5 seconds This clearlyshows how quickly a developer can test the developed pilot services on a local emulator instanceIt also gives the developer early feedback and insights eg to know which of the NSs will needmore time to be instantiated The full evaluation that gives more insights into the behavior of thesmart manufacturing pilot running on-top of the emulator is published in [39]
Wrapper for SONATA SP
As part of its final release the SONATA Service Platform is extended with a wrapper for theEmulator making it possible to use the SP to orchestrate on the emulated PoPs of the emulatorThe target of this effort is to make the SP usable in a local setup so that SP-associated workflowssuch as Service and Function Specific Managers can be tested locally as well This feature is ofparticular interest for the specific manager testing framework which will use it to create a localtest environment for specific managers A more detailed description of this Emulator wrapper canbe found in Deliverable D52 [23]
42 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 313 On-boarding and instantiation performance of an early version of 5GTANGOrsquos smartmanufacturing pilot on top of the emulator [39]
3111 Benchmarker
The tng-sdk-benchmark tool is a novel tool introduced into the 5GTANGO SDK in release 50 Itis based on the ideas and concepts of the old son-profile tool initially presented in [42] Howevertng-sdk-bechmark or tng-bench in short is a complete re-write that includes the lessons learnedfrom the initial design Tng-bench aims to be a framework for the end-to-end automation of VNFbenchmarking processes Its goal is to automate the benchmarking process in such a way thatVNF performance profiles can be generated without further human interaction This enables theintegration of VNF benchmarking into continuous integration and continuous delivery (CICD)pipelines so that new VNF profiles are generated on-the-fly for every new software version of aVNF
The work done in relation to this tool also contributed to one of the main standandisationactivities of 5GTANGO which can be found in the IETF draft about ldquoMethodology for VNFBenchmarking Automationrdquo [10] Tng-bench is one of the two reference implementations currentlylisted in that IETF draft Its design was presented at the IETF 104 in Prague in March 2019 tothe IETF benchmarking methodology working group
Scope
One of the problems in automated NFV deployments is that relatively small changes in the code of aVNF or service might have unforeseen large impacts to the resulting performance of the productiondeployment Another problem of upcoming NFV systems is that the automated management ofservices eg based on machine learning techniques needs insights about the performance behaviorof the involved VNFs and services prior their deployment To solve these issues a tool is neededthat allows developers to quickly test their VNFs and services in a wide variety of configurations(eg resource assignments) to learn about the behaviour of the developed artefacts This exper-imentation process has to be fully automated to be able to integrate it into the CICD pipelinescommonly used in DevOps setups
tng-bench provides exactly this and represents a benchmarking experiment automation frame-work for NFV developers It offers a rich description approach for benchmarking experimentswhich can then be automatically executed by tng-bench With this it can also be used as fully-automated data mining tool to generate performance datasets which can then be picked up by other5GTANGO tools eg tng-sdk-analyze to gain deeper insights into the performance behavior ofVNFs and services
5GTANGO Public 43
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 314 High-level architecture artifacts and workflows [34]
31111 Release 50
Release 50 is the first release in which tng-sdk-benchmark is shipped as part of 5GTANGOrsquosSDK As a result no change log is given since all developments can be considered as new
31112 Architecture
Fig 314 shows the high-level architecture artifacts and workflows of tng-bench as we alsopresented it in [34] Each tng-bench setup consists of two main components Tng-bench and oneor multiple NFV platforms to execute the actual experiments ie deploy and run the VNFs andservices under test (SUTs) Both components should run in separated environments eg on twoseparated physical machines and tng-bench must be able to connect to the execution platform tocontrol and monitor them
A typical workflow to benchmark a given VNF or service looks as follows First a user (eg de-veloper) specifies a performance experiment description (PED) which is a YAML document thatdescribes the entire experiment eg in terms of configurations to be tested and references a5GTANGO package that contains the SUT Once this document is created it is together withthe SUT package given to tng-bench which then reads it and starts the benchmarking process(Fig 314 s1) In the next step tng-bench explores the complete configuration space that shouldbe tested ie it computes the Cartesian product of all configuration options and number of ex-periment repetitions specified in the PED Once this is done the different configurations (whichcan be thousands) are applied to the descriptors of the PED For example new descriptors aregenerated in which 1 vCPU is assigned to a SUT VNF another is generated with 2 vCPUs and soon In addition probe VNFs are added to the SUTs as it is specified in the PED Those probes cancontain eg traffic generators used to stimulate the SUT during the experiments All these newconfigurations and probes are then used to generate a series of new 5GTANGO service packagesone for each configuration to be tested (Fig 314 s2)
Having those SUT packages generated tng-bench enters the next phase in which it starts toactually on-board those packages on the connected execution platforms and deploy them one aftereach other (Fig 314 s3) After a new SUT package is deployed (SUT and probes are instantiated)
44 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
tng-bench instructs the probes to start the experiment eg to generate traffic This executionphase runs for a fixed amount of time as defined by the PED eg 60 seconds (Fig 314 s4) Duringthis tng-bench collects monitoring data from the execution platforms SUT and probes and storesit (Fig 314 s5) Once the experiment runtime is over the SUT is terminated and deleted beforethe next SUT package containing another configuration is deployed This process continues untilevery configuration package has been deployed and tested once and all results and monitoring datais collected
Finally the resulting data is combined into multiple tables distinguishing between experimentresults and time series monitoring data and written to disk From there it can be picked up by the5GTANGO Analysis Engine (Sec 3112) for further analysis
Experiment Definition Model
To automate benchmarking experiments using tng-bench a user (eg developer) has to specify a socalled performance experiment descriptor (PED) that defines the experiments eg configurationsto be tested and references the SUT A PED is a YAML file that follows a pre-defined YANG datamodel which defines the structure and fields that must be used The reason why this data model isdefined using YANG instead of JSON schema like the other 5GTANGO models is that the modelis part of our IETF standadzation effort [10] and the IETF strictly requires YANG models
Fig 315 shows parts of the generated tree of the PED YANG model Each PED file can containone or multiple performance experiment descriptions each having a unique name (id) On top eachdescription can be identified with a vendor name and version triple to make the model compatibleto all other 5GTANGO data models Next a PED references one or more target SUTs again using5GTANGOrsquos vendor name version identification approach It also contains a URI field whichallows to reference a 5GTANGO package that contains the VNF or service to be tested (SUT)
After that the actual experiment is define in the experiments section Each experiment has aunique ID and some options like number of repetitions and a time limit that defines how long eachconfiguration should be tested Next each experiment can define a list of probes which are CNFsthat are deployed side-by-side to the SUT and used to stimulate the SUT eg by generating trafficThose probe definitions contain the names of the containers to be used as well as the identifiers ofthe connections points of the SUT to which they should be connected Optionally multiple networkconfigurations can be specified
Finally a list of parameters is defined for each experiment Each of these parameters referencesa particular VNF CNF or probe of the SUT to which it will be applied during experiment ex-ecutions Among others the parameters are cmd-start and cmd-stop to specify start and stopparameters applied to the SUT or the probes as well as resource configuration parameters likecpu-core-set cpu-bw mem-max and so on Each of these parameters can be defined using asingle value (scalar) a list of values to be tested (vector) or a range of values to be tested definedby begin end step (loop) This feature allows to efficiently specify complex parameter studieswith small efforts Tng-bench takes care to execute every possible combination of these parameterswhen an experiment is executed
31113 Installation
The installation of tng-bench itself is simple and can be performed together with the installationof the other 5GTANGO SDK tools as described in our quick guide [38] However to actuallymake use of the tool an execution platform needs to be installed and configured to be used Weprovide a detailed online documentation to perform this setup and provide Ansible playbooks toautomatically install an execution machine using vim-emu as execution platform [12]
5GTANGO Public 45
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 315 Part of the YANG model specifying the format of the performance experiment de-scriptors (PED)
46 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
31114 Usage
The following listing shows the synopsis of all CLI arguments supported by tng-sdk-benchmark
release 50
$ tng-bench -h
usage tng-bench [-h] [-v] [--loglevel LOG_LEVEL] [--logjson] -p PED
[-c CONFIGFILE] [--work-dir WORK_DIR] [-rd RESULT_DIR]
[--no-generation] [--no-population] [--no-execution]
[--no-result] [--validation] [--hold]
[--max-experiments MAX_EXPERIMENTS] [--no-display]
[--generator SERVICE_GENERATOR] [--ibbd IBBD_DIR] [-y]
[--no-prometheus]
Manage and control VNF and service profiling experiments
optional arguments
-h --help show this help message and exit
-v --verbose Increases logging level to debug
--loglevel LOG_LEVEL Directly specify loglevel Default INFO
--logjson Use 5GTANGO JSON-based logging Default False
-p PED --ped PED PED file to be used for profiling run
-c CONFIGFILE --config CONFIGFILE
Config file to be used eg defining the execution
platformsDefault configyml
--work-dir WORK_DIR Dictionary for generated artifacts eg profiling
packages Will use a temporary folder as default
-rd RESULT_DIR --result-dir RESULT_DIR
Dictionary for measured results eg logfiles
monitoring data Default rsquo(cwd)resultsrsquo
--no-generation Skip profiling package generation step
--no-population Skip experiment population step
--no-execution Skip profiling execution step
--no-result Skip result processing step
--validation Skip all package validation steps
--hold Stop when experiment is started and wait for user
input (helps for debugging)
--max-experiments MAX_EXPERIMENTS
Maximum number of experiments to generate irrespective
of PED def (helps for debugging)
--no-display Disable additional outputs
--generator SERVICE_GENERATOR
Service configuration generator to be used Default
rsquoeu5gtangorsquo
--ibbd IBBD_DIR Dictionary for generated IETF BMWG rsquobenchmarking
secriptorsrsquo Default None
-y --force-yes Answer all user questions that might appear with yes
--no-prometheus Do not launch Prometheus automatically
5GTANGO Public 47
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 316 Prometheus dashboard showing the execution of multiple experiment rounds
Figure 317 Example of a time series metric recorded during a single experiment round
Example Results
We performed some test experiments using the Suricata IDS VNF which is also used in the emulatorversion of the Smart Manufacturing pilot We tested this VNF with different workloads basedon recorded traffic traces During the experiment the VNF was configured with 1600 differentconfigurations eg CPU times from 10 to 100 and 256MB as well as 512MB memory TheVNF was also tested with different rule sets
During the experiment tng-bench sequentially deployed the VNF 1600 times on the vim-emutest infrastructure Each of these deployments was then executed for 60 seconds before it is ter-minated and the next configuration is deployed This results in a total experiment runtime of 12hours which can be seen in Fig 316 The figure shows tng-benchrsquos Prometheus database usedto record time series metrics during the experiments It nicely shows how the performance of theVNF changes between the different experiment executions with stepwise increasing CPU time as-signments The figure also gives an impression on the amount of data points which can be quicklymined in a completely automated fashion using tng-bench
Fig 317 in contrast shows an example plot of a single time series metric recorded during one
48 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
60 second experiment round It shows the CPU utilization of the VNF container (vnf0vdu010)and the probe containers (mpinput and mpoutput) over time The VNF container was limitedto 20 in the depicted configuration Each container was pinned to its own CPU core Thefigure shows how the VNF containerrsquos resources can be limited by tng-bench to emulate differentruntime situations and resource configuration ie the measured CPU utilisation does not exceedthe used configurations Please note that this figure is only an example and many more time seriesmetrics can be collected during an benchmarking experiment Time series results like these canthen be used by 5GTANGOrsquos Analytics Engine eg to detect correlations between configurationparameters and performance metrics Those insights are not only useful for the VNF and servicedevelopers but also for operations eg to guide automated MANO solutions in the resourcedimensioning process
3112 Analytics Engine
The Analytics Engine supports the realisation of various analysis processes targeted to the ex-traction of insights and profiles in VNF and NS level mainly with regards to resource usage andelasticity aspects The supported analyses may be realised in an experimental or operational con-text Under this perspective the Analytics Engine can realise analysis based on results comingfrom the benchmarking tools or based on results coming from the tests realised within the VampVIn the first case the analysis results are mainly given as feedback to software developers in order toidentify performance issues capacity limits and bottlenecks in the provided software (eg a VNF)and proceed to corrective actions or appropriately dimension the requirements for the efficientdeployment and operation of the software In the second case the results can also lead to thedesign and specification of effective policies (eg elasticity policies) or the incorporation of machinelearning models for forecasting purposes In the current deliverable focus is put on the first case
31121 Release 50
The Analytics Engine is a new component that is included in the SONATA Release 50 Thus nochange log is available for this release The main supported functionalities are
bull selection of monitoring metrics and time period for input dataset
bull fetch time series data from the Monitoring Engine (eg a Prometheus instance)
bull execution of an analysis process
bull presentation of results in the form of a URL
31122 Architecture
Within the 5GTANGO SDK the Analytics Engine (tng-analytics-engine) is interconnectedwith the benchmarking tool (tng-sdk-benchmark) Upon the execution of a set of experimentsthe provided data is stored as time series data and exposed as raw data as well as data integratedinto a Prometheus instance Through the APIs provided by the Analytics Engine these data canbe considered as input data for the execution of an analysis As analysis results the outcomes areprovided in the form of a series of URLs and made available to application developers as shownin Fig 318 It should be noted that details regarding the internal architecture of the AnalyticsEngine and the supported APIs is detailed in [22]
5GTANGO Public 49
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 318 Profiling Sequence
31123 Usage
An API has been designed and implemented for realising an analysis upon getting access to thedata generated by experiments realised with the Benchmark tool given that they are made availablein a Prometheus instance By using this API an analysis can be initiated over data collected fromthe available benchmarking results The start and end time of the time series data the set ofconsidered monitoring metrics in the analysis and the type of the analysis service (algorithm) tobe executed are declared through this API
bull REST - API Endpoint httpanalytics engine server IP addressprofiling service
bull POST body parameters
start The datetime that the experiment(s) was initiated
end The datetime that the experiment(s) was terminated
name String with the name of the analysis service to be executed (eg linear regression)
step The frequency used for getting data from Prometheus This is an optional field
metrics A JSONArray with the set of metrics for which time series data is going to be fetched This is an optional field
dimensions A JSONArray with the dimensions to be considered per metric This is an optional field
metrics-without-dimensions JSONArray This is an optional field
metrics-keyword JSONArray This is an optional field
An indicative analysis for a linear regression is defined as follows
start2019-03-04T073030781Z
end2019-03-04T173030781Z
50 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 319 Correlation analysis of Suricata VNF
step10s
name linearRegression
metrics [mn_mp_output_vdu01_cpu_stats__online_cpus_intns_idns-1vnf-ids-suricata experiment_idsuricata_performancemn_mp_output_vdu01_networks__eth0__rx_bytes
ns_idns-1vnf-ids-suricataexperiment_idsuricata_performance]
The outcome of an analysis is usually a URL however it can be also combined with raw outputdata for further processing
[
httpopencpu_serverocputmpx0d8b61dcbe8022console
httpopencpu_serverocputmpx0d8b61dcbe8022filesfinaldatacsv
httpopencpu_serverocputmpx0d8b61dcbe8022filesmetricsCombinationhtml
]
Indicative Analysis Results
As a first use case the Analytics Engine has been used for the realisation of resource efficiency andelasticity efficiency analysis in the 5GTANGO smart manufacturing pilot Specifically performanceprofiles of the Suricata IDS VNF and the MQTT VNF have been produced following the availabilityof results upon a set of experiments realised within the benchmark tool
Specifically correlation analysis has taken place for the identification of correlation amonginfrastructure-oriented and VNF-specific metrics (see Fig 319 and Fig 320) Based on the pro-vided results the corresponding linear regression models have been produced
For instance in Fig 321 the linear regression model produced concerning the relationshipbetween the Suricatarsquos CPU usage and decoder bytes metric is provided Given that Suricatarsquosprocessing is highly related to the incoming traffic a trend for CPU usage increase is shown uponrelevant increase in the processed bytes
5GTANGO Public 51
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
Figure 320 Correlation results in table format
Figure 321 Linear regression model for Suricatarsquos CPU usage and decoder bytes metric
52 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
32 External Interfaces
In contrast to 5GTANGOrsquos VampV or SP 5GTANGOrsquos SDK is not deployed as an online serviceInstead it is a loosely coupled set of tools that can work together but can be all also used in astandalone setup In addition those tools are usually installed on a local development machinewhere they are accessed through command line interfaces Because of this the amount of fixedAPIs offered by the SDK is limited Still some components offer external interfaces because theyare for example also deployed as helper functionality in the VampV or SP [20]
321 Components with external interfaces
The components that offer a external API are listed in the following Each of them has its own APIspecification mentioned and referenced in the corresponding component section of this document
bull tng-vnv-dsm (Sec 313)
bull tng-sdk-project (Sec 314)
bull tng-sdk-package (Sec 315)
bull tng-sdk-validation (Sec 316)
bull tng-sdk-analyse (Sec 3112)
bull vim-emu (Sec 3110)
322 5GTANGOrsquos advanced package format as main interface
In fact 5GTANGOrsquos advanced package format can be considered as the main ldquointerfacerdquo betweenthe 5GTANGO SDK and other NFV ecosystems This is not limited to 5GTANGOrsquos VampV andSP but also extends to third party platforms like OSM mdash one of the key factors to increase there-usability of 5GTANGOrsquos SDK
The reason for this is that almost all artifacts that can be created with the SDK including VNFsservice compositions tests FSMs and SSMs are packaged into 5GTANGO packages before theyare on-boarded and uploaded to the VampV SP or other platform This results in file-orientedinterfaces offered by the target platforms allowing the upload of package files (tgo) The SDKin turn ensures that it is always able to generate standards-compliant packages [16]
We presented this workflow at IEEE NFV-SDN 2018 targeting not only the 5GTANGO SP butalso OSM rel FOUR [40] Further details about APIs of these platform can be found in D33 andD52 [22 23]
5GTANGO Public 53
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
4 Final release features
Release 50 of the 5GTANGO SDK is more than a simple successor of SONATA Release 40The entire SDK toolset has been extended with 5 strong tools focusing on improved support fortesting benchmarking and profiling while improving usability through a user-friendly portal anddecision support engine to propose relevant tests and VNFs during the development process Inaddition the existing toolset has been significantly improved through new cloudnative supportsupport for SLAs and policies and a range of minor improvements and bugfixes Many of thesenovel developments have been driven by the (common) needs of 5GTANGO pilots As before alldeveloped components in Release 50 are open-source and support public feedback through issuereporting and pull requests Below we recapitulate the highlights of the latest and final releasewithin the scope of 5GTANGO
Table 41 Final release SDK highlights (new components in bold)
SDK tool Release 50 highlights
schema Schema definitions define all descriptor formats and now support cloud nativeand physical deployment units as well as QoS requirements
developer portal Graphical User Interface to trigger any of the SDK tools in support of aseamless development workflow Can be deployed remotely or local
decision support engine Component which proposes tests andor VNFs based on a user profile and itssimilarity to other users
descriptor generator Template generation tool which integrates with the project management tooland SDK portal through its extended REST API
packager Package creation tool for different MANO platforms supporting several storagebackends (eg tng-cat OSM ONAP) extended REST API and CLI
validator Syntactical and semantic checking tool of descriptors of VNFs and servicesProvides support for cloud-native functions SLAs policies slices and testdescriptors Support for custom rules
sm tester Tool to assist testing of service specific managers (SSMs) and enabling reusablepatterns for state migration merging and splitting
test creation framework Tool enables functional testing of services and VNFs using Python Developedtests can be executed locally on the emulator and reused on the VampV
emulator Tool supporting local deployment of NFV services under development Includessupport for cloud-native NFs and multi-VDU deployments support for E-LineE-LAN and E-Tree networks support service function chaining and support forCICD with OSM
benchmarker Tool which automatically enables the generation of performance profiles under arange of pre-defined (resource) configurations Output is integrated with theanalytics engine
analytics engine Performance and correlation data analysis supporting time-based selection ofmonitored metrics time series analysis and extensive visualization of NFVservices
54 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
5 Pilot Requirements
The main driver for the developments performed for the SDK in the final iteration of this projecthave been the three pilots Communications Pilot Immersive Media Pilot and Smart ManufacturingPilot (see [21] and [24] for more details on pilots) This section intends to highlight the mappingbetween the requirements coming from the pilots and the final release features showing this strongrelationship between WP4 and WP7
Table 51 Pilot Requirements vs Final Release Features
SDK functionality Communications Pilot Immersive Media Pilot Smart Manufacturing Pilot
Project managementamp creation
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
Pilot relies on the canonicalSDK workflow andassociated tools for projectmanagement
- Portal Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
Pilot creators novel to theSDK need assistance intriggering the right SDKtools
- QoS support (schema) Pilot requires strict latencyjitter and throughput
Throughput guaranteesneeded
Latency requirements
- Descriptor generation Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
Pilot has many VNFs andinitial descriptor templatesare needed
- Cloud-native design(schema SM state)
Adequate resiliency toguarantee sufficientconnectivity
Pilot needs ability to scaleefficiently in all video-relatedfunctionscontainers
Scaling support neededSession state exists in IDSand FW NFs and requireadequate handling duringscaling failover events
Testing- Descriptor validationand customization
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
Wide range of NFs anddescriptors requirevalidation Customizationneeded
- Ad-hoc testing(emulation)
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing in localenvironment will increaseconfidence before going toproduction
Ad-hoc testing is stronglyrequired to assessintegration latency andfunctionality of the manycomponents
- SM testing SM to support failovermechanisms needs to belocally validated
SMs to support scalingmechanisms need to belocally validated
SMs to support scaling andfailover mechanisms need tobe locally validated
- Functional testautomation (creationand execution)
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Many service-level tests needto be re-evaluated uponevery development change
Performanceevaluation- Benchmarking Performance evaluation of
QoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
Performance evaluation ofQoS needs to be evaluated inmany scenarios
- profilinganalysis Correlation and bottleneckanalysis needed to high QoS
Correlation and bottleneckanalysis needed to ensurehigh throughput
Correlation and bottleneckanalysis needed to ensurelow latencies
The following sections describe why and how the three pilots make use of different parts ofthe SDK Please note that some of the SDK components for example the packager or validator
5GTANGO Public 55
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
are used for every pilot since they are required for main steps in the integrated development of5GTANGO
51 Communications Pilot
The Communication pilot has been developed with VNFs adapting an existing real product com-mercially deployed using pre-built VMs In order to carry out this adaptation it was necessary todo an extensive use of all the SDK tools from the creation of the VNFDs with tng-sdk-project
to the management of the packages with tng-sdk-package
Besides this pilot uses new features such as SNMP monitoring or QoS flavors which are includedin the SLAs For all these file descriptors it is essential to ensure their validity and correctnessbefore uploading them to the platform Thus the tng-sdk-validation tool has been speciallyhelpful to write the descriptors of all the components involved whose complexity has increaseddue to the use of these newly introduced features The inclusion of the validation with customrules allows the user to set predefined requirements that will be checked and ensured all along thedescriptor generation process
52 Immersive Media Pilot
The immersive media pilot has hybrid network services consisting of both VNFs and CNFs Thevalidator and packager are extensively used to verify correctness of definitions of such complexnetwork services and to prepare them for uploading and deployment The content managementsystem is a core component of the service tng-sdk-sm and tng-sdk-benchmark are used to analyzethe performance and create and test appropriate FSMs and SSMs to ensure the service adequatelyreflects increasing traffic load and changing conditions
The pilot also involves completely containerized functions The SDK Emulator is used to testsuch functions locally in an interactive manner Test creation framework tng-sdk-test is used tocreate automated functional tests for these network functions and to prepare test packages for theVampV platform
53 Smart Manufacturing Pilot
The smart manufacturing pilot is entirely implemented with CNFs instead of legacy VNFs Tothis end it directly depends on the changes introduced in the 5GTANGO descriptor schemas Inparticular the smart manufacturing pilot was the first proof of concept of a CNF using multipleCDUs This is for example used by the cloud connector (CC) CNF which consists of four CDUsSimilarly the pilot depends on the new features introduced into the project manager packager andvalidator which now all come with support for CNFs as well as multi-CDU descriptions
Another outstanding example of the smart manufacturing pilot driving the SDK implementationsis vim-emu In fact the first version of the pilot was entirely implemented and tested on the vim-emuplatform To do so the platform and in particular the 5GTANGO LLCM needed to be extendedwith support for CNFs and multi-CDU descriptors Further a series of features like port mappingsand support for multiple service instances was added to the LLCM in order to fully support thispilot This workflow showed great benefits since it allowed to quickly prototype the componentsinvolved in the pilot in a local environment It was in particular helpful for custom CNFs like themachine data collector (MDC) or CC which had to be developed from scratch for the 5GTANGO
56 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
project Finally this lightweight prototyping platform will be used to demonstrate the pilot to abroad public audience at IEEE NetSoft in June 2019
Finally the pilot was used to run first tests of the benchmarker component on real-world CNFsleading to a series of data sets used for further evaluation eg published in the upcoming deliver-ables and scientific publications
5GTANGO Public 57
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
6 Next Evolution
The current 5GTANGO SDK is the result of several years of innovating development on a toolsetand methodology for NFV services The SDK originated from the SONATA project and wassignificantly extended and improved in the 5GTANGO project The SDK triggered competinginitiatives such as OSM and ONAP OSM adopted several components from the 5GTANGO SDKincluding its emulator and its packaging functionality The SDK of ONAP was only proposed in2017 once SONATA already had such a framework unique among open-source MANO frameworks
Despite the wide range of improvements and newly introduced components the 5GTANGOfinal release does not necessarily stop development and maintenance once the 5GTANGO projectends In addition to potential bug fixes the 5GTANGO SDK toolset enables further extensionstaking into consideration novel technology andor market trends Below some potential futuredirections are sketched out These might be targeted through individual developers novel researchand innovation projects or other instances
61 Descriptors schema generation packaging and validation
Despite the efforts of ETSI different NFV platforms still rely on different structuresschemas forthe descriptors involved 5GTANGO largely complies with the former however for a wide rangeof reasons (eg pilot requirements) extensions or modifications were required Similar evolutionsare observed on other platforms Such a situation is a logical consequence of introducing the novelNFV paradigm into practice We therefore anticipate still a period of further diversification inthe data models until at some point all existing efforts can be compared and mapped in order toformulate a converged format At that point in time it will make sense to adapt the 5GTANGOSDK toolset to such a converged format
The generation of descriptors in 5GTANGO is assisted through the automatic descriptor gen-erator Rather than relying on a particular format of descriptors based on assumptions of theunderlying service a catalogue of use-case specific templates might be made in order to enableeven further reducing time and error-proneness of descriptor writing focusing on the characteris-tics of particular use cases
5GTANGO provides a unique descriptor validation tool enabling customisation of the validationlogic This enables adaptation to the specific requirements of different use cases (eg verticals)Additional checks related to the definition of different flavours can be considered The main goalof these tests would be to detect flavour duplications and inconsistencies in the descriptors Thisproblems could be reported in some cases as warnings especially in the case of formal inconsistencieswhich could be intended for specific use cases
Before services described as a set of descriptors could actually be deployed they need to bepackaged The 5GTANGO packaging tool is unique in its capability to support multiple platforms(incl 5GTANGO OSM and ONAP) Its extensible design allows to even further extend and alignit with future specifications published by TOSCA and ETSI
58 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
62 Development workflow and portal
As documented earlier in this deliverable 5GTANGO provides a set of development tools whichcould be combined in different ways We identified a canonical development workflow and thecorresponding SDK portal fits in this philosophy It could be considered to even further integratethe different SDK tools within the portal in order to provide a truly Integrated DevelopmentEnvironment (IDE) for NFV development Such an environment could include other tools likecatalogues SDK test deployment on emulation etc
63 Local testing and analysis
The local testing abilities of the current 5GTANGO SDK release are considerable It includes anemulator for ad-hoc testing a test framework for automating function test creation recommenda-tion and execution and a benchmarking and analytics framework for performance benchmarking
Future efforts could focus on the automated execution of recurring tests for services Similarto reactive Integrated Development Environments (IDEs) such frameworks could automaticallyexecute tests upon save and provide feedback through dialog windows or log files In addition therecommendation engine could be further enhanced with information of different selections of theusers (such as SLAs or Policies) Thus providing more user targeted test recommendations Theemulator already integrates well with OSM however automated testing on top of OSM or otherplatforms could be further developed
The performance of software-based services as in NFV is an ongoing research topic The bench-marking and analysis tools of 5GTANGO provide unique tools to assist in the performance analysisand improvement of NFVs In order to further assist the development design and evaluation ofperformance models open data sets for the NFV community are important as this enables them toevaluate new schemes on similar data sets Such data sets could build further on the initiatives ofthe SNDZoo project [36] Models and analysis schemes themselves as part of the analytics enginemight be further developed in order to assist the developer in finding the most important bottle-necks or enabling them to intra- or extrapolate trends based on limited measuring data relyingon machine learning models The analytics engine itself might be further extended to supportalternative programming languages to formulate analysis and modelling schemes
5GTANGO Public 59
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
7 Source Code and installation
Release 50 continues 5GTANGOrsquos approach to open-source all developed tools and componentsThe SDK has created separate repositories for all individual discernible components All reposi-tories are hosted within the SONATA GitHub organisation [15] The components list shown belowprovides an overview of the mapping of each component to the repository it makes use of
SDK toolCode repository (undergithubcomsonata-nfv) Short description
schema tng-schema 5GTANGO descriptor record and packagespecifications and schemas (data models)
developer portal tng-portal SDK portalproject creator tng-sdk-project tool to manage network service projectsdescriptor generator tng-sdk-descriptorgen tool to rapidly create descriptors based on high-level
objectivesimage converter tng-sdk-img tool to rapidly create VIM-dependent imagesdecision supportengine
tng-vnv-dsm tool to assist in the recommendation of appropriatetests or VNFs during the composition process
sm tester tng-sdk-sm tool to rapidly testgenerate specific managers (egSSM FSM)
packager tng-sdk-package tool to create package from project foldervalidation tng-sdk-validation tool to validate file descriptors from the projectVIM emulator vim-emu (at OSM [33]) tool to deploy services locally on a Mininet-based
environmentbenchmarker tng-sdk-benchmark tool for fully automated VNF and network service
benchmarking and profilinganalytics engine tng-analytics-engine tool to analyze produced monitoring data eg as
generated from the benchmarking tooltesting framework tng-sdk-test framework to create and execute functional tests
71 Installation
Although the reader might be interested in the use of individual SDK components and thereforecan rely on the installation instructions of those components individually (either as provided earlierin this document or as available on the GitHub repositories) for many other uses it might bepreferable to install the entire 5GTANGO SDK at once For this purpose we recommend readersto follow the official online guide that provides the most recent installation instructions [38]
60 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
8 Conclusions
This deliverable reports on the novel developments and workflows introduced in the 5GTANGOSDK release 50 The focus of this release is on the support for cloud-native design improved testingsupport and extensibility towards OSM ONAP and potentially other platforms The documentaims to be largely self-contained however many pointers to up-to-date information are provided tothe corresponding GitHub repositories or 5GTANGO website regarding installation instructions
The resulting 5GTANGO SDK consists of 14 components (bundled in 13 repositories) which canbe executed largely independently or in a typical workflow as described in the architecture sectionThe SDK portal assists users through the entire development process and provides an interface ableto launch individual SDK tools from a single frontend The linear workflow consists of componentimplementation (supported by 3 tools) composition of components (through 4 tools) bundling andpackaging (via 2 tools) validation and (functional) testing (using up to 4 tools) and benchmarkingand analysis focusing on performance (based on 3 tools)
Many of the newly introduced tools andor functionalities were triggered through requirements ofthe three pilots in 5GTANGO Although most of the SDK functionalities are usable across pilots thecommunications and smart manufacturing pilot introduced support to define and evaluate QoS andperformance-related aspects while the immersive media pilot posed some particular (functional)testing-related requirements Among many other functionalities these have been accommodatedby the test creation and execution framework as well as the benchmarker and analytics engine
The 5GTANGO SDK is one of very few efforts in the NFV ecosystem assisting service developerswith an extensive toolkit and workflow Its modular and open-source design however encouragesextensions beyond the scope of 5GTANGO A number of pointers with respect to this goal weredocumented Beyond these potential pathways the tool authors will maintain the respective codebases until and beyond the end of the 5GTANGO project ensuring usability stability and sufficienttraction in the NFV world for the near and further future
5GTANGO Public 61
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
A Bibliography
[1] 5GTANGO tng-vnv-dsm API Online at httpssonata-nfvgithubiotng-docurls
primaryName=5GTANGO20VNV20Recommendation20Engine20API20v1
[2] 5gtango tng-vnv-dsm github Online aturl[httpsgithubcomsonata-nfvtng-vnv-dsm
[3] Angular Website Online at httpsangulario
[4] Json schema Website Online at httpjson-schemaorg
[5] Kubernetes Website Online at httpskubernetesio
[6] Pytest Online at httpspytestorg
[7] Sonata project Website Online at httpwwwsonata-nfveu
[8] tng-sdk-test documentation Online at httpstng-sdk-testreadthedocsioen
latestindexhtml
[9] tng-sdk-test examples Online at httpsgithubcomsonata-nfvtng-sdk-testtree
masterexamples
[10] Methodology for VNF Benchmarking Automation Internet-Draft draft-rosa-bmwg-vnfbench-02 IETF July 2018 Work in Progress
[11] Git Website 2019 Online at httpsgit-scmcom
[12] 5GTANGO 5GTANGO tng-bench Execution Platform Install Instructions2019 Online at httpsgithubcomsonata-nfvtng-sdk-benchmarkwiki
Setup-execution-platform-(vim-emu)
[13] 5GTANGO 5GTANGO tng-sdk-sm Repository 2019 Online at httpsgithubcom
sonata-nfvtng-sdk-sm
[14] 5GTANGO Project D22 Architecture Design Nov 2017 Online at 5gtangoeudocumentsD22_v1pdf
[15] 5GTANGO Project 5GTANGO GitHub Organization Mar 2018 Online at https
githubcomsonata-nfv
[16] 5GTANGO Project 5GTANGO Package Specification Jan 2018 Online at githubcom
sonata-nfvtng-schemawikiPkgSpec_LATEST
[17] 5GTANGO Project 5GTANGO SDK Packager API Mar 2018 Online at httpsgoogl6GMt2h
[18] 5GTANGO Project 5GTANGO SDK Validator API Mar 2018 Online at httpsapp
swaggerhubcomapis-docsQuobistng-sdk-validation-swagger100
62 Public 5GTANGO
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[19] 5GTANGO Project 5GTANGO Service- and Function Specific Manager APIMar 2018 Online at httpsgithubcomsonata-nfvson-mano-frameworkwiki
Service-and-Function-Specific-Managers
[20] 5GTANGO Project D41 first open-source release of the sdk toolset Apr 2018 Online at5gtangoeudocumentsD41_v1pdf
[21] 5GTANGO Project D71 Evaluation strategy and testbed setup report Jun 2018 Onlineat https5gtangoeuproject-outcomeshtml
[22] 5GTANGO Project D33 Verification and validation platform final release May 2019 Onlineat https5gtangoeuproject-outcomeshtml
[23] 5GTANGO Project D52 Service platform final release May 2019 Online at https
5gtangoeuproject-outcomeshtml
[24] 5GTANGO Project D72 Implementation of pilots and first evaluation Feb 2019 Online athttps5gtangoeuproject-outcomeshtml
[25] A Karatzoglou B Hidasi Recurrent neural networks with top-k gains for session-based rec-ommendations In Proceedings of the 27th ACM International Conference on Information andKnowledge Management 2018
[26] L Mei et al An attentive interaction network for context-aware recommendations In Proceed-ings of the 27th ACM International Conference on Information and Knowledge Management2018
[27] ETSI Open Source MANO (OSM) Website 2019 Online at httpsosmetsiorg
[28] Thomas Spetzier Gerd Breiter Frank Leymann TOSCA Cloud Service Archive (CSAR)Website July 2012 Online at httpswwwoasis-openorgcommitteesdownloadphp
46057CSAR20V0-1docx
[29] Docker Inc Dockerhub Build and ship any application anywhere Online at hhttpshubdockercom
[30] Docker Inc Docker An open platform for distributed applications Website August 2013Online at httpwwwdockercom
[31] ETSI European Telecommunications Standards Institute ETSI GS NFV-SOL004 WebsiteJuly 2017 Online at httpwwwetsiorgdeliveretsi_gsNFV-SOL001_09900402
0301_60gs_nfv-sol004v020301ppdf
[32] ONAP Open networking automation platform Website August 2017 Online at [https
wwwonaporg](httpswwwonaporg)
[33] Open Source MANO OSM vim-emu Website 2019 Online at httpsosmetsiorg
gitwebp=osmvim-emugit
[34] M Peuster and H Karl Profile Your Chains Not Functions Automated Network Service Pro-filing in DevOps Environments In 2017 IEEE Conference on Network Function Virtualizationand Software Defined Networks (NFV-SDN) Nov 2017
5GTANGO Public 63
Document 5GTANGOD42Date June 5 2019 Security PublicStatus To be approved by EC Version 10
[35] Manuel Peuster Stefan Schneider Daniel Behnke Patrick-Benjamin Boek and Holger KarlPrototyping and demonstrating 5g verticals The smart manufacturing case In IEEE 5thConference on Network Softwarization (NetSoft) IEEE 2019
[36] Manuel Peuster Stefan Schneider and Holger Karl The softwarised network data zoo arXivpreprint arXiv190504962 May 2019
[37] rdquoThe OpenStack Projectrdquo Openstack The open source cloud operating system WebsiteJuly 2012 Online at httpwwwopenstackorg
[38] 5GTANGO project consortium 5GTANGO Quick Guide Online Documentation 2019 Onlineat httpssonata-nfvgithubio
[39] Stefan Schneider Manuel Peuster Daniel Behnke Patrick-Benjamin Boek and Holger KarlPutting 5g into production Realizing a smart manufacturing vertical scenario In IEEEEuropean Conference on Networks and Communications (EuCNC) IEEE 2019
[40] Stefan Schneider Manuel Peuster Wouter Tavernier and Holger Karl A fully integratedmulti-platform nfv sdk IEEE NFV-SDN 2018 2019
[41] SONATA Project D31 Basic SDK Prototype May 2016 Online at httpwww
sonata-nfveucontentd31-basic-sdk-prototype
[42] SONATA Project D33 SONATA SDK final release Website 2017 Online at http
sonata-nfveudeliverables
[43] Faqir Zarrar Yousaf Michael Bredel Sibylle Schaller and Fabian Schneider Nfv and sdnkeytechnology enablers for 5g networks IEEE journal on Selected Areas in Communications35(11)2468ndash2478 2017
64 Public 5GTANGO
- List of Figures
- List of Tables
- Introduction
-
- Document scope
- Overview
-
- Cloud-native support
- Validation verification and testing
- Extensible design and support for alternate platforms
-
- Document structure
-
- 5GTANGO Development and Testing Lifecycle
-
- Phase 1 Development and Testing using the SDK
- Phase 2 Validation and Verification using the VampV Platform
- Phase 3 Deployment and Execution using the Service Platform
-
- Architecture
-
- Components
-
- Schema for Descriptors
- SDK Portal
- Decision Support Engine
- Descriptor generator and project management
- Packager
- Validator
- Testing framework
- Development and testing of specific manager functionality
- State management support
- Emulator
- Benchmarker
- Analytics Engine
-
- External Interfaces
-
- Components with external interfaces
- 5GTANGOs advanced package format as main interface
-
- Final release features
- Pilot Requirements
-
- Communications Pilot
- Immersive Media Pilot
- Smart Manufacturing Pilot
-
- Next Evolution
-
- Descriptors schema generation packaging and validation
- Development workflow and portal
- Local testing and analysis
-
- Source Code and installation
-
- Installation
-
- Conclusions
- Bibliography
-