the most important thing we build is trust - testforce.com · the most important thing we build is...

NFV enabling automation and performance validation for Network Services and Labs

The most important thing we build is trust

White Paper

© 2016 Cobham - 2 -

NFV automation frameworks for performance validation of network functions and services

1 The impact of NFV .................................................................................................................................... 3

1.1 NFV driving transformation..................................................................................................................................3 1.2 Demystifying MANO ............................................................................................................................................4 1.3 NFV driving transformation in labs ......................................................................................................................6

2 NFV architecture - enabling automation for validation ......................................................................... 7

2.1 SuperCloud Introduction .....................................................................................................................................8 2.2 TeraVM Introduction ...........................................................................................................................................9 2.3 Advantages of OpenVIM as an automation framework for validation ...................................................................9 2.4 SuperCloud NFVO/VNFM ................................................................................................................................. 12

3 Validation methodology for MANO ....................................................................................................... 13

3.1 REST API validation approach .......................................................................................................................... 13

3.2 Stateful awareness of MANO API responses .................................................................................................... 14 3.3 Variance between VIM API query structures ..................................................................................................... 14

3.4 Requirement for an application aware validation solution .................................................................................. 16 3.5 Noisy neighbor scenario .................................................................................................................................... 18

4 Delivering validated NFV network services .......................................................................................... 19

4.1 Onboarding validation ....................................................................................................................................... 19

4.2 VNF Lifecycle management and orchestration validation .................................................................................. 23 4.3 TeraVM enabling automated performance validation of VNFs ........................................................................... 25 4.4 VNF validation - from lab to production networks .............................................................................................. 27

5 Network Service validation and performance monitoring .................................................................. 28

5.1 Correlation of performance validation data ........................................................................................................ 30

6 Conclusion .............................................................................................................................................. 32

6.1 OSM - enabling an NFV automation framework for validation ........................................................................... 32 6.2 Delivering performance and secure NFV frameworks........................................................................................ 33

© 2016 Cobham - 3 -

NFV automation frameworks for performance validation of network

functions and services

1 The impact of NFV

Network function virtualization (NFV) is revolutionizing both how carriers enable network services

and how network solution vendors approach and develop network products. With carriers seeking

to deliver new network services to their customers in weeks rather than months, carriers are

aggressively pushing solution vendors to deliver more in shorter time frames.

To meet these aggressive time-lines lab automation is now a critical component to the success of

NFV. The goal of this application note is to enable both carriers and solution vendors to make an

informed decision on what automation means in terms of delivery of a NFV enabled network service.

But it also shows how automation for NFV can enable automation solutions in our labs today.

The ETSI ISG NFV (http://www.etsi.org/technologies-clusters/technologies/nfv), is one of the

driving forces behind recommending how carriers and solution vendors approach the future of

networking. A key take-away from their published documentation is that the old ways of delivering

product and the automation processes currently used need to change. This disruptive

transformation means that for the test and measurement industry that a new paradigm of delivering

test solutions is now upon us.

1.1 NFV driving transformation

Progressive carriers are at an inflection point, in which they no longer see or accept the need for

proprietary hardware and/or vendor solution lock-in. The challenge for many, is how now to adopt

and be commercially successful in this new technology shift, one in which agility and flexibility are

core. Many carriers now see that open source has some of the answers and are looking to

standards bodies to move at the same speed. This melding of open source and standards is

producing viable solutions, albeit multiples with a similar goal.

It’s clear that no two carriers are the same and will take a different approach to NFV, for example

this will include the core technology choices for the management and orchestration (MANO) stack.

For network solution vendors and test solution vendors, the challenge is to be able to provide

solutions which are open and agile, which are agnostic to their surroundings i.e. can easily fit into

these new automation frameworks i.e. NFV automation lifecycle wheel of: onboarding, cataloging,

orchestration, deployment and service chaining.

http://www.etsi.org/technologies-clusters/technologies/nfv

The impact of NFV

© 2016 Cobham - 4 -

1.2 Demystifying MANO

The ETSI ISG NFV has delivered multiple recommendations covering the key topics of: architecture

design, management flows, orchestration principles, performance management, test and security

best practices. If we focus on the management and orchestration (MANO) recommendations, it’s

now possible to deliver automation for lab environments at minimal costs.

The vision from the carriers is that the next generation of networks will be open, in that the chosen

components can easily be swapped out. This principle is a driving force behind a number of open

source communities e.g. OpenStack, OpenMANO, OPNFV, etc.

The NFV recommendation for the MANO provides for a modular structure of three key layers, each

module providing a level of abstraction enabling simplification of use for the carrier network

operations, essentially removing the need for highly skilled personnel to deliver a custom network

service. The modules outlined in the recommendations address:

NFV Orchestration (NFVO) – Onboarding and catalogue management of virtual network

functions (VNF)

VNF Management (VNFM) – Management of the lifecycle of the VNF i.e.

update/upgrade/etc

Virtual Infrastructure Management (VIM) – Resource manager for compute, storage and

network

To enable the abstraction between modules requires an open application programming interface

(API) interconnecting the modules. With many communities and vendors now electing to use

Representational State Transfer (REST), which is simply a client to server communication using

the HTTP protocol to communicate instruction sets.

MANO and the proposed capabilities are a critical piece of the carriers NFV vision. This means that

for network solution vendors to prove that their solutions are indeed NFV ready, they must deliver

or push the VNF through the same information or objective modelling process and automation

process flow that the carrier have deployed.

The impact of NFV

© 2016 Cobham - 5 -

The advantage of doing so provides automation to validate packaging, onboarding capabilities,

catalogue presentation of metadata, accuracy of deployment scenarios and of course performance

in the custom network service.

Figure 1: ETSI ISG NFV example architecture

The impact of NFV

© 2016 Cobham - 6 -

1.3 NFV driving transformation in labs

For many to prove NFV readiness and to automate labs for a NFV process flow; is challenging and

potentially cost prohibitive. The fact is - it does not have to be and in this paper we look at using

open source components to deliver aspects of the MANO architecture.

For example a simple and easy to use virtual infrastructure resource manager is available as part

of the Open Source MANO (OSM) project supported by a number of global carriers which include

Telefonica, BT, Telenor and Telecom Austria. [https://osm.etsi.org/]

The OpenVIM component of OSM provides a number of advantages which are highlighted later,

but fundamentally it provides for a stable foundation to learn and understand about how services

and network functions can be deployed using information models.

A further observation is that the software being promoted across the NFV community, is the

similarity of the automation functionality to that of proprietary lab automation solutions, especially

in terms of cataloging, management of deployments and resource management.

With so much of the software required for automation now readily available in the open source

communities, the advantage of adopting an NFV approach is that: vendors can prove flexibility of

VNFs to work in numerous architectures/platforms, with the agility to create custom platforms which

accurately represent the carrier NFV lifecycle wheel.

Figure 2: Using an automation framework e.g. OSM for VNF lifecycle validation

NFV architecture - enabling automation for validation

© 2016 Cobham - 7 -

2 NFV architecture - enabling automation for validation

As a starting point for this application note, it’s worth defining a sample hardware platform

architecture. The network function virtual infrastructure (NFVI) i.e. where the network service will

run. The platform utilizes standard hardware blade servers (fundamental to the NFV vision) and

also uses commodity network interface cards, hence minimizing the costs of building out scaled

labs.

An open and standard operating system such as RedHat or Ubuntu is used to deliver the core

operating system, on which the hypervisor is installed. In this instance the Kernel based Virtual

Machine (KVM) with Quick Emulator (QEMU) provides the virtualization layer. The popular virtual

machine management library (Libvirt) provides a standard interface to assist in managing the virtual

machine (VM) instances.

For networking, the open libraries for open vswitch are installed along with the OpenFlow controller

enabling centralized management for networking. This approach enables ease of configuring and

managing the associated network service (NS) VNF networking topologies, which can be defined

as an information model such as a VNF forwarding graph (VNF-FG).

Taking a base NFV architecture enables us with basic validation methodologies, which include a

validation methodology for APIs in use in the NFV framework. API validation techniques are

important, as it enables us to determine how information modelling impacts the different levels of

the MANO stack architecture. Also by selecting the right test methodology early on enables us to

adopt a one armed approach for validation i.e. by selecting the right test tool, the easier it is to

cover multiple test scenarios in the NFV project lifecycle.

In the true spirit of NFV, a number of components from unique sources are used to create a

reference architecture, which can be easily replicated by the reader. Both components are

discussed in detail throughout the application overview.

Components in use:

1. OpenVIM from the OpenMANO project.

2. SuperCloud (VNFM and NFVO) provided by Luxoft


© 2016 Cobham - 8 -

2.1 SuperCloud Introduction

SuperCloud will act as the NFVO/VNFM providing:

1. API stability – fixed API implementations e.g. orchestrator and VIM (Or-Vi)

2. Integration capabilities – enabling performance validation results from TeraVM to be

exposed alongside key performance indicators for host, VNF and NS SLA.

3. Catalogue Management – begin to standardize the VNF descriptor files

4. VIM agnostic: ability to swap between VIMs e.g. OpenStack/OpenVIM without losing

functionality of information modelling

5. Fault, Configuration, Accounting, Performance, Security (FCAPS): A single pane of glass

view for performance monitoring of the NFV platform.

Figure 3: Abstracted NFV platform architecture

The architecture above demonstrates and proves the ability to mix both open source and

proprietary solutions in the MANO stack. The architecture design enables users to support multiple

VIM configuration options e.g. OpenVIM or OpenStack, whilst maintaining the integrity and

functionality of the NFV lifecycle flow process.


© 2016 Cobham - 9 -

2.2 TeraVM Introduction

TeraVM is an application emulation and security performance solution, delivering comprehensive

test coverage for application services, wired and wireless networks. TeraVM is offered as a

virtualized solution enabling the flexibility to run anywhere - lab, datacenter and the cloud, with

consistent performance coverage, ensuring that highly optimized networks and services can be

delivered with minimal risk.

TeraVM offers a wide range of application emulation and performance validation for system under

test (SUT) ideal for VNFs, physical appliances and/or hybrid deployments which may use cloud

bursting techniques.

2.3 Advantages of OpenVIM as an automation framework for validation

OpenVIM is an open source project which is supported by Telefonica, OpenVIM is designed to

enable ease of use. It enables use of open vswitch and Openflow controllers to deliver networking

and management, ideal for information modelling of networking. The networking functionality is

similar in concept to that of the OpenStack Neutron module, but today that’s where the similarity

ends. OpenStack Neutron has become highly complex to both configure and operate, however the

OSM has adopted a different approach and are looking to keep networking as simple as possible.

Noticeable in OpenVIM is the simplicity and ease of ability to quickly tune the VNF deployment for

performance, enabling the use of performance tuning techniques such as non-uniform memory

access (NUMA). The simplicity is enabled in part by the information model scheme.

In OpenStack tuning is not easily achieved, requiring multiple adaptions on the host and tenant

setup e.g. (configure cpu allocation ratios, modify scheduler configurations, etc).*1

Another feature of OpenVIM which is very relevant when it comes to delivering high performance

on the dataplane: is the ability to deploy components to use pass through techniques (Direct-Path,

SR-IOV, etc) ensuring the solution vendor can achieve maximum throughput in the physical

network.

The OSM project is about simplicity and ease of deployment but by focusing on the key

performance tuning attributes enables solution vendors to quickly assess multiple deployment

options i.e. orchestrate VNF with/without NUMA.

*1 https://www.sdxcentral.com/articles/contributed/dimensioning-openstack-neutron-nfv-systems-mark-lambe/2014/09/

https://www.sdxcentral.com/articles/contributed/dimensioning-openstack-neutron-nfv-systems-mark-lambe/2014/09/


© 2016 Cobham - 10 -

2.3.1 VNF packaging in OpenVIM

Frustratingly for carriers, they are inundated with vendors proposing solutions that support NFV,

when all they merely offer is a software package and instruction set on how to run the software on

a VM with prescribed additional package installs. The whole purpose of NFV is to deliver to the

carriers a packaged solution which can use information or objective models in order to create a NS.

The VNF package consists of the network function’s machine image (in KVM known as QEMU

Copy On Write or qcow) and descriptor files for deployment. The associated descriptor files or

information model can be segmented into a generic VNF descriptor (VNFD) and VNF deployment

unit descriptor (VDU). OpenVIM supports ingest of this metadata in its north-bound API.

For solution vendors, this enables a viable methodology for their development teams to learn about

the VNF onboarding process and to create accurate information models of the VNFD and VDU.

With minimum effort the development team can quickly validate the accuracy of the VNF

deployment.

Furthermore, for the development and operations team - OpenVIM’s RESTFul API makes it ideal

to run deployment exercises without the need of any sophisticated user interfaces.

An example onboarding exercise could be achieved using a browser e.g. Firefox with RESTClient

(http://restclient.net/) plugin, in which users can quickly start interaction with the infrastructure

manager (a full list of API calls is available on the OpenMANO github repository). However, it is

worth noting that these tools enable an easy interface to debug issues, post deployment.

As we touch on the REST API, a number of recommendations are available in ETSI ISG test

documentation, where there is a need to validate performance on these APIs. In the next chapter,

we will discuss a test methodology, to enable an understanding of the performance of the MANO

stack and the associated REST API calls.

http://restclient.net/


© 2016 Cobham - 11 -

Figure 4: RESTClient a Mozilla Firefox plugin


© 2016 Cobham - 12 -

2.4 SuperCloud NFVO/VNFM

SuperCloud provides an integrated solution acting as a NFVO/VNFM module. As the abstraction

module above the OpenVIM it provides an easy to use framework which enables deployment of

network services.

In addition to enabling the upper modules of the MANO stack, a key advantage of the solution is to

provide a single pane of glass view on the performance of the running NFV platform and on each

of the unique network services running. SuperCloud provides for key performance indicators

covering compute, networking and tenant SLAs.

SuperCloud key capabilities include:

Administration functions (complete view of infrastructure and tenants)

Tenant Management (incl. security, bandwidth allocation)

Abstraction of network complexity (network topology and network creation)

VNF Catalogue Management (complete view of available VNF packages)

VNF onboarding/deployment (ingest VDU & VNF descriptors along with VM images)

VNF Manager (lifecycle upgrade/update management)

Performance monitoring (performance metrics covering – host, VNF and NS SLA)

Figure 5: SuperCloud UI – Topology implementation screen

Validation methodology for MANO

© 2016 Cobham - 13 -

3 Validation methodology for MANO

Clearly a number of solutions exist in the open source community for test capabilities, however

each are niche and most are not application aware. TeraVM as a stateful application emulation

provides a one armed approach, enabling carriers and network equipment vendors with significant

test coverage. In the following sections, we look at a number of test methodologies which are

unique to TeraVM.

3.1 REST API validation approach

As discussed in the previous chapter, the use of an API based on REST requires careful

consideration. As each layer of the MANO stack has its own instruction set, this means that there

is a translation required of the input data on the north-bound API to the south-bound API. This

abstraction translation consumes compute cycles and adds latency.

For example – “What are the differences in performance between two VIM APIs e.g. OpenStack

and OpenVIM for particular jobs – deploy a VM with NUMA assignment?

As both VIMs have their own unique instruction to achieve the correct positioning of the VM, and

for most cases the positive test is deployment where the resources are available. However, what

happens where NUMA cannot be achieved?

For example a stateless test tool will present to the user a latency response to indicate that it took

x-milliseconds, but in fact it’s possibly a failure response. Understanding what the meaning of the

responses incoming on the API is key to accurate validation.

A further observation from a carrier perspective, is that the API versions supported between MANO

modules is dynamic i.e. no two are the same. This translates in terms VIM types as generic and

specific VNFM types. To re-iterate the point VIM types will have unique instruction sets e.g. to

achieve VM deployment with NUMA. Test tools must be flexible enough to support these granular

differences.

A first step in the validation methodology is to determine performances in the API, this will cover

Instruction set timing performance

Ability to handle hundreds of concurrent instructions

Responsiveness under duress


© 2016 Cobham - 14 -

3.2 Stateful awareness of MANO API responses

The key to API testing is stateful awareness of previous instructions, forming a type of feedback

loop. The feedback loop must be intelligible enough to translate the business logic being returned,

as part of the HTTP responses to be part of the next set of instructions to be sent.

Again in terms of API selection, simply saying that it is RESTful is one consideration, the other is

the format of the information being communicated as part of the API request/response. Both

OpenStack and OpenVIM have elected to use JavaScript Object Notation (JSON), enabling ease

of presentation of parameters and values.

In terms of test solution vendor selection, the tool must be layer 7 aware i.e. can parse data and

use it as part of the next request. This stateful awareness is available in TeraVM, further

distinguishing it from open source tools.

3.3 Variance between VIM API query structures

For carriers to validate performance of the MANO stack requires that the test and measurement

solution be able to handle numerous query types. Highlighted in the following example are the

HTTP request constructs for OpenStack and OpenVIM

OpenStack uses security toked-ids, the token-id is created as part of an initial HTTP GET request.

This token-id is specific to a tenant username/password login post. The token is used to validate

user requests, but also helps the VIM to deliver data specific to that tenant.

OpenVIM (at the time of writing has no security token-id implementation) query structure uses a

tenant-id, which is extracted from a separate HTTP GET request.


© 2016 Cobham - 15 -

OpenStack

HTTP Header:

X-Auth-Token = {security.token.id}

Content-type: application/JSON

Command: GET

Link: http://<IP_address_OpenStack_Image_API>/v2.0/images

OpenVIM

HTTP Header:

X-Auth-Token = {not implemented at time of writing}

Content-type: application/JSON

Command: GET

Link: http://<IP_address_OpenVIM>/openvim/<tenant_ID>/images

Note the highlighted text in the request urls for both VIMs. The query structure are different. Clearly

this is where an agnostic VNFM adds value as it can translate “Show me images for my tenant”, to

each of the relevant VIM APIs.

It’s also worth noting that the JSON response from both VIM types are different. This does highlight

the challenges of working with a truly open NFV framework including a generic VNFM at this time.

Alignment of the northbound API requests and responses in the VIM need further consideration

and is a core objective for many of the working groups at the ETSI ISG NFV today.


© 2016 Cobham - 16 -

3.4 Requirement for an application aware validation solution

Test and measurement solutions must be agnostic to the NFV automation framework, in that it

should be capable of delivering an instruction set or multiple commands on the API irrespective of

VIM type. However and more importantly it must be application aware.

As highlighted already in the previous sections, it is not as simple to say that a single query will get

you the <parameter>:<value> pairing you seek.

When choosing a solution to validate MANO ensure that the test and measurement solution is

application aware i.e. must be able to build multiple queries based on previous responses received.

The following is an example request configuration for API performance validation in OpenStack.

1. Enable security token-id via a HTTP POST, the body contains the user details: "auth": { "tenantName": "demo", "passwordCredentials":{ "username": "demo", "password": "DEMO_PASS"} }

2. Parse response to extract security token-id

"token": { "issued_at": "2016-01-26T11:54:54.906705", "expires": "2016-01-26T12:54:54Z", "id": "3afdd03757774071bd14b28046ba95e6", "tenant": { "description": "Demo Project", "enabled": true, "id": "3150b91202b94e0cb6e6b7300a38c834", "name": "demo" }, "audit_ids": [ "IksRwnKGTyOPCpWkFg0XZA" ] }

3. Use token-id to create the next requests

NOTE: A key advantage of TeraVM is it’s per flow architecture which means that it can deliver

the granularity necessary to facilitate validation with multiple request instruction sets, but more

importantly offers the scale necessary to enable validation as if hundreds of tenants are active

on the NFV framework.


© 2016 Cobham - 17 -

3.4.1 VIM API performance validation methodology

The following is an example test methodology which is used in validating VIM responsiveness to

an instruction set. The example uses a simple request to list available images per tenant. The

instruction set requires a number of HTTP requests. Note the methodology can be equally applied

to any of the MANO layers e.g. NFVO -> VNFM, VNFM ->VIM.

As each REST request is unique, carriers must assess each step to understand where potential

compute cycles are intensive and slow, emphasizing the need that validation requires the

granularity of performance measurements per request. A key advantage of using Cobham Wireless

TeraVM, is the unique ability to create multiple requests and to measure performance on the

granularity of per request.

Unique to TeraVM is the ability to parse responses from the incoming JSON file enabling a series

of stateful requests, enabling carriers/solution vendors to quickly pin-point bottlenecks.

The following is an example test methodology which can be used for VIM performance validation.

1. Concurrently connect to all REST API / admin interfaces: e.g. Identity, Compute, Image a. Assess memory and CPU usage

2. Assess timing between different VIM API layers a. Assess latency for Identity/Compute/Image

3. Security – service multiple tenant logins/UUIDs a. Access/Authentication flood of UUID requests on the identity database

4. VM launch - how many VMs can the VIM instantiate a. Assess concurrent instruction handling versus a step approach of one after the

other

5. Host management – will the VIM scale quickly a. Extend NFVI, who quickly will the VIM recognize resources and assign new VMs

6. Robustness – hundreds of tenant instructions a. Identify memory leaks, caching issues


© 2016 Cobham - 18 -

Shown below is an example performance validation report based on the command for deletion. In

this simple test case a single REST DELETE request is presented on the API, the results from the

server is that it takes up to 200ms for the server to respond as instruction completed (HTTP code:

200).

Figure 6: Performance validation of a delete instruction on a REST API

3.5 Noisy neighbor scenario

Understanding the API validation methodology clearly has advantages, but more significantly

enables the NFV platform to be robustly tested with real scenarios. A one armed test approach,

means that the stateful API calls, can be used to create unique scenarios like the noisy neighbor.

From a carrier perspective this could mean turning up unique network service topologies which

could mean moving large VNF image files from repository to the deployment compute node,

which can be compute intensive.

Again only by emulating multiple tenant use cases i.e. tenant 1 – with an active network service

and tenant 2 – turning up a service with large VMs, is it possible to determine the impact between

tenancies and more importantly the live service.

Delivering validated NFV network services

© 2016 Cobham - 19 -

4 Delivering validated NFV network services

This section is aimed at understanding the NFV lifecycle process to enable the network service and

more importantly how it drives a pre-production network deployment validation methodology.

As part of the understanding of how to deliver a comprehensive validation methodology, this section

also provides some recommendations on requirements in creating a NFV validation lab: enabling

efficient management and testing of available VNFs for solution vendors.

4.1 Onboarding validation

Onboarding is the process of taking the external VNFs info the NFV framework and managing them

in a catalogue. The key for all VNFs - is to ensure that the package to be onboarded includes the

VNF images and the necessary descriptor files, which are ingested by the orchestrator. The data

is used by the VIM for the successful deployment of the VMs to the compute node and enabling

the networking required for an operational VNF.

An example VNF onboarding scenario could be traffic as a service – TeraVM. When implementing

the information model or descriptor metadata for TeraVM for OpenVIM a number of positive

observations were made, which are worth sharing here for other VNF solution vendors.

The descriptor files enable flexibility, for example a VM image (with the correct drivers) can be

consumed in multiple ways. That is one descriptor file may define connectivity to a vswitch and the

other could be used to define a choice of pass through technology. For TeraVM this provides for

ease of deployment to test inside a tenant and across hosts or multi-domain tenancies. Unique

descriptors can also enable provisioning of different NUMA requirements.

The clever piece for the VNF vendor is that it references the same VM image, stored in a VNF

package repository. From a carriers perspective this offers a number of efficiencies, most obvious

being storage. For users of TeraVM, it means that they can deployment options are available which

provides for greater test coverage, again a key differentiator to open-source and competing tools.


© 2016 Cobham - 20 -

Figure 7: SuperCloud onboarding flow chart for OpenVIM

Another important part of the VDU and VNFD is the enablement of the VNF lifecycle management

process. Over the life of the service a number of updates/upgrades will be made on individual

running virtual machines. A flexible and well maintained VNF package repository will contribute to

the success of the NS, this is one of the core features of a good NFVO solution such as SuperCloud.

At the early stage of the NFV process cycle, solution vendors must understand how the VDU

descriptors impact the performance of their solutions, if it is badly thought out or described it can

seriously hamper the carrier experience. Clearly as a first impression, solution vendors should

provide VM images and descriptor files (information models) that match the carrier onboarding

process, the importance of which is often under-stated.


© 2016 Cobham - 21 -

4.1.1 Considerations for VNF licensing validation

From an automation perspective for a network service or even facilitating performance testing, the

descriptor files provide other relevant information which can be used for accurate cataloging of the

VNFs available in the NFV platform. Versioning and licensing details are key parameters which

need to be included in the descriptor files. Knowing your VNFD and VDU is fundamental, simply

being - if it doesn’t work e.g. incorrect licensing details/location, the carrier simply can’t use the

VNF.

4.1.2 Example information modelling of a VNF (TeraVM)

The following is a snapshot of a TeraVM descriptor file, which is delivered in JSON format. JSON

provides a human readable parameter:value pairing with minimal text, ensuring ease of creation

and consumption of the information needed for a VNF deployment.

######

# Example VNF descriptor file in JSON format used as part of the onboarding process for

# the TeraVM traffic as a service VNF.

######

{

"id":"Cobham_TeraVM_Executive-1.0-353",

"version":"12.01_353",

"service_type":"service_verification_executive",

"service_instance_type":"tvm-e",

"license":"10.16.122.34:8989",

"name":"Cobham TVM-E VNF",

"description":"Cobham TeraVM executive VNF",

"author":"Cobham Wireless",

"shared":true,

"tenant_id":"t1",

"vnfc_list":{

"1":{

"name":"TVM-E",

"author": "Cobham Wireless",

"version": "12.01-353",

"license": "licence-server",

"description":"TeraVM executive VNF",

"min_instances":1,


© 2016 Cobham - 22 -

"max_instances":1,

"vdu_list":{

"TVM-E_12.01-353":null

},

"intf_list":{

"1":{

"name":"Comm",

"description":"TVM internal communication interface",

"vdu_intf_id":{

"TVM-E_12.01-353":1

},

"access":"bidirectional",

"intf_class":"external",

"min_instances":1,

"max_instances":1

}

}

}

}

}

NOTE: TeraVM is a fully virtualized solution which is packaged for numerous hypervisors which

include ESXi, KVM and Xen. Enabling carrier and solution vendors validate performance with

consistency and ease across multiple platforms. In addition TeraVM supports public cloud platforms

of AWS and Azure, with support for private cloud platforms such as OpenStack and OpenVIM.


© 2016 Cobham - 23 -

4.2 VNF Lifecycle management and orchestration validation

The following sections provide will outline in detail the orchestration aspects of a NS, a significant

piece of NFV, but it is worth noting some of the fundamentals that are needed for VNF lifecycle

management. *2

As already highlighted an advantage of keeping control of an accurate VNF catalogue, is that it

ensures that the process for upgrade/update can be implemented without complexity. As vendors

release new patches, it will be necessary to run the update/upgrade but more importantly to easily

snapshot the running VM and catalogue it as a new/unique VNF version in the repository.

In selecting the correct NFVO/VNFM pairing e.g. SuperCloud, there should be the functionality to

enable a roll back service, ensuring prior to any update/upgrade, the deployed environment can be

restored to an operational state.

4.2.1 Orchestration flow process

A significant requirement for NFV is the use of information modelling not just at a VNF level, but

implementation of the network service. Below diagram steps through a process of deploying a

network service, in the section that follows

Figure 8: Example deployment scenario using the MANO stack

*2 IEEE publication: Realizing network function virtualization management and orchestration with model based open architecture

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7367393

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7367393


© 2016 Cobham - 24 -

4.2.2 Network Service topology creation

Once the new network service or lab deployment requirements have been pieced together as per

the example in Figure 8, the next step is to transfer the business plan into a working topology. At

this point there is a translation of the commercial requirement to a technical topology.

A number of tools are available today that implement to some degree this functionality, this is where

there is some similarity to traditional automation tools in the lab i.e. a wizard enabling translation of

commercial requirements to a deployable topology. But this is where the similarity ends, as the

NFVO must now correlate what the carrier request is and present the requested VNFs and

descriptor files to the VIM, along with required networking links to enable the network service chain.

SuperCloud NFVO provides a mature interface enabling ease of use in translating business needs

to technical deployments, which provides a very precise topology, quickly and easily via a drag and

drop user interface.

A feature of SuperCloud is the ability to create catalogues of service topologies, meaning network

function vendors can create backlog catalogues, so at any time they can turn up an archived VNF

version for test in a single click, ideal for regression testing.

Figure 9: SuperCloud convert business plans into deployable network services

The advantage of the SuperCloud solution is that the user does not have to completely understand

the service chain and/or component VNFs, enabling ease of use for either IT or networking folks.


© 2016 Cobham - 25 -

4.3 TeraVM enabling automated performance validation of VNFs

To this point we have discussed the features of OSM and NFV and its clear using open source we

can deliver an automation framework supporting information modelling and ease of deployment via

a business logic using simple to user interfaces as provided in SuperCloud.

The assumption at this point is that the network service is operational, so the next logical step is to

begin to validate performance of the network service. Clearly from an automation perspective, it

would be weak to expect that the carrier/VNF solution vendor would need to go through a manual

process to deliver performance or service assurance validation.

A clear advantage of selecting a solution like TeraVM, is that it:

1. Aligns to the NFV vision for information modelling

2. Maps to OSM core principle of simplicity i.e. template driven with minimal user

configuration.

Figure 10: TeraVM simplifying validation through centralized and templated validation use

cases


© 2016 Cobham - 26 -

Highlighted in the above figure is the TeraVM centralized library approach for validation. This

provides carriers/solution vendors with an integrated approach to validate the widest range of

network service use cases e.g. security, video delivery networks, vEPC, etc, each unique network

service can be validated with a pre-defined library of validation use cases.

The advantage of the centralized library is the ability to schedule and deliver into the automation

framework at any stage the required validation use case. Adopting a centralized library approach

ensures repeatable and reliability in assessing performance.

Clearly this a differentiator between TeraVM and open source test tools. Furthermore by

centralizing golden use cases as templates minimizes the need for manual configuration by users.

This is critical for repeatable testing across multi-domain tenancies and/or test teams.

For example let’s consider a highly specialized test methodology for a NS enabling a mobile virtual

network operator (MVNO), the requirement is such that the carrier can lock in place the core

parameter configuration for the radio access networks, gateways, management entity and

continually thereafter be able to validate key performance indicators.

The key is that the instance of the test solution itself is ephemeral, turned up for a duration in the

tenancy and is removed once KPIs are gathered. However, the test cases themselves are

persistent ensuring consistency in validation for the future.

NOTE: TeraVM provides performance validation of key application services of voice, video and data. TeraVM offers validation of core mobile networks (vEPC), security validation of both access technologies (VPN) and security appliances. TeraVM’s comprehensive cybersecurity threat library is used in security hardening of major network services.


© 2016 Cobham - 27 -

4.4 VNF validation - from lab to production networks

The telecoms industry prediction is for exponential growth which will come from the likes of the

Internet of Things and the subscriber’s insatiable demand for video. These challenges multiplexed

with the NFV principles of mobility and portability means that the VNF solution vendors are facing

an unprecedented challenge – the ability to prove scale.

Today the core networks are handling terabits of data. Scaling is a key test case that both carriers

and solution vendors need to perform for carrier grade NFV, however in doing so this poses a

number of open challenges that warrant validation:

1. Robustness of the MANO stack to manage racks of compute, with NS under real load

2. Performance monitoring across multi-host/tenant domains

3. Continuous security assessment (keeping the bad guys out)

This further compounds the needs for a scalable virtualized validation solution, such as TeraVM.

More importantly it highlights the constraints that open source or indeed proprietary hardware

based test solutions exhibit. For both the carriers and solution vendors to deliver a test bed to cover

all scenarios may appear daunting. Again by choosing the right test vendor partnership, it’s possible

to deliver validation at scale more importantly cost effectively.

TeraVM can deliver the multiple validation use cases for NFV at scale. TeraVM’s approach is to

use an elastic test bed on standard hardware. One of the advantages to this approach, is that, once

the initial validation at scale is completed, the underlying infrastructure can be released back into

compute pool and the TeraVM can be used in smaller test beds for dev-ops as per the earlier

validation methodologies described in the NFV lifecycle process.

This approach provides carriers and network function vendors with massive savings in capital

expenditure.

.

NOTE: As TeraVM is a virtualized solution, it means it can run on the carriers chosen NFVI. For

example once the scaled performance validation is complete, the infrastructure can be released

back into the pool, to be used as a part of a service offering for a customer. TeraVM at any point

there-after can be turned up as and where needed, at the necessary scale, per available resources.

Network Service validation and performance monitoring

© 2016 Cobham - 28 -

5 Network Service validation and performance monitoring

Fault, Configuration, Accounting, Performance and Security (FCAPS) are key to carrier-grade NFV.

This is where the combined solution of TeraVM and SuperCloud leaves the open source and other

solution offerings behind.

As indicated a comprehensive NFV test methodology must encompass how the end-to-end network

service is validated, but more importantly can be extended to monitor operational performance in

real-time. So when moving from lab to live deployments this should be achieved with: minimal

intervention from the operator and at minimal costs.

So far we have seen how TeraVM is used to deliver a one armed test approach to environment

and deployment scenarios, used to assess a range of service validation use cases, providing the

carrier/solution vendor with confidence in the different service type offerings. However, once the

NS goes live the job switches to performance monitoring and service assurance.

The advantage of the SuperCloud solution is the ability to clearly see a number of key performance

indicators at various levels which includes the NFVI, VNF and indeed across the MANO stack. The

solution provides for fault notification, enabling carriers/solution vendors to enable event logging

when things are not right in the NFV framework.

Core to the success of NFV will be the ability to quickly view performance and more importantly

trigger and alert when things get hot, that could be at a host compute level, traffic bottlenecks, in a

particular VNF instance or indeed the poor quality of service could be experience at Layer 7.

SuperCloud provides this visibility but more importantly it’s possible to create thresholds to trigger

warnings, these triggers can then be used to deliver an automated test use case from within

TeraVM. The combined solution offers greater visibility on where the pinch-points are.


© 2016 Cobham - 29 -

As NFV matures and carriers choose their preferred automation frameworks, the move from

validated proof of concepts in labs to live network deployments will happen swiftly. Service

assurance will be key, visibility is required for KPIs across the platform.

Figure 11: SuperCloud integrated performance monitoring window

In the instance where the network service is experiencing issues, TeraVM can be triggered to

assess performance from within the tenant or compute domain. Again in this instance TeraVM

can deliver a simplified business logic view of things, through the template driven UI. Carriers are

enabled with simple pass/fail like criteria, enabling accurate judgement of where things are going

wrong.


© 2016 Cobham - 30 -

5.1 Correlation of performance validation data

As already outlined a templated approach with a simplified view on KPIs, melds with the experience

that carriers and vendors expect for NFV.

In the example validation methodologies highlighted in this tutorial, we spoke of a one armed

approach to the validation methodology, now we must take that one armed approach to interpreting

and understanding the returned performance metrics.

If we take the noisy neighbor scenario, it is unique in its requirements i.e. there is loading validation

ongoing on an active network service and there is a validation of the neighbor activity e.g.

instantiating the second network service.

What is not obvious to this point is that there are two unique KPIs being delivered concurrently,

both independent of each other but yet inter-linked:

1. Performance of the active network service under load

2. Performance of the API calls of the NFV framework

In tenant 1: the reporting view could be simply be a line graph e.g. a throughput validation

assessment, but key is that view should have the ability to highlight if some violation occurs e.g.

Dropped Packets > X%.

In tenant 2: the reporting view is of a business logic e.g. time and status of an instruction being

implemented in the REST API.

So from a validation perspective it makes it possible to correlate the performance between the two

unique validation steps and more importantly determine is one aspect of the NFV framework impact

on another.

This is a unique advantage of the TeraVM, during run time users can switch between the two unique

views of the running validation scenario, with an easy to interpret view, but more importantly the

results or KPIs are exported as one entity enabling ease of analysis.

The core belief for NFV is to deliver agile and flexible network services cost efficiently, this must

also translate to test and measurement. But by choosing the right solution for performance and

validation today i.e. TeraVM in SuperCloud, ensures you the reader have made the right decision

and it will help maximize the return on your investment for many years to come.


© 2016 Cobham - 31 -

Figure 12: TeraVM real time key performance indicators and violation presentation screen

Figure 13: TeraVM business logic enabled performance measurement screen

Conclusion

© 2016 Cobham - 32 -

6 Conclusion

In this application note we set out to show how NFV can enable automation frameworks which can

be used to validate network functions and network services, but more importantly offers the reader

ideas on how open source MANO frameworks can offer cost efficiency for validation. The

application note also showed by selecting the right solution partners that it’s possible to deliver and

cover a range of validation scenarios for the lab, but more importantly that the right solution can

also be brought to the live network service to act as a quality assurance solution, maximizing the

return on investment for years to come.

6.1 OSM - enabling an NFV automation framework for validation

The premise for NFV is to enable carriers with agility and flexibility. The telco-industry recognizes

that to enable the NFV principles that there is now a melding of open source initiatives with

standards, with the aim of advancing the carrier’s needs more rapidly and in a sustainable manner.

One such community is Open Source Mano (https://osm.etsi.org/).

From a test and measurement perspective these very same guiding principles are enabling many

advantages and not just from a cost perspective. The approach in these communities is to ensure

complexity is abstracted enabling simplified and easy to use automation frameworks.

The core principle of openness for management and orchestration has resulted in numerous open

source communities providing technology to achieve automation. For network function vendors this

provides a significant cost saving in lab automation for validation. For the carrier it provides a

simplified approach to ensuring service assurance.

Open source means both vendors and carriers can easily access and take in-house a base solution

for an automation framework, with minimal risk and zero cost. From a vendor perspective this

means they can easily replicate a potential carrier’s environment. By replicating the carrier

environment and using the same information modelling and automation processes, the solution

vendors can ensure there development and operations are in-line with their carrier customer,

throughout the complete lifecycle of the virtual network function and/or network services.

Conclusion

© 2016 Cobham - 33 -

6.2 Delivering performance and secure NFV frameworks

However, by choosing the right NFV framework is critical for carriers, one which is reliable, robust

and performing. Today there are many variants of frameworks available, but for the network

solution vendor it means they need to be agile and flexible in the delivery of their solutions. A key

strategy for many vendors is to support multiple hypervisors or deliver containerized solutions,

along with enabling a flexible molding to support information or objective modelling of the network

product.

The dynamic and changing landscape of NFV is resulting in new challenges, key being the ability

to prove that virtual network functions are ready for live deployment, compatible with a multitude of

carrier NFV frameworks and support the VNF lifecycle – onboarding, deployment and reliability at

the necessary scale.

With so many unknowns on the plate, the aim for all vendors/carriers should be for a consistent

test and measurement approach using known and proven solutions. This means investment in the

right validation solution, such as TeraVM. Of course, the open source domain produces solutions

for network validation, but the use of such solutions adds yet another unknown into the mix,

heightening the risk for failure. A core value add for electing TeraVM earlier on, is the ability to

easily integrate security validation use cases, from secure access to cybersecurity threat mitigation.

More importantly once the NFV framework is live there is the need to assure that the cyber-criminals

are kept off the NFV platform by running regular security checks and screening.

NFV validation is multi-dimensional i.e. from information modelling process validation, to validation

of management APIs and assuring the reliability/robustness/security of a tenant architecture or

Network Service. By selecting the right validation partner from as early as possible in the NFV

project, assures that the widest range of validation scenarios are covered, ensuring reliability over

the life of the tenant service and most importantly tenant security.

By selecting the right validation partner today, maximizes the overall return on investment in

validation for many years to come.

NFV automation frameworks for performance validation of network functions and services

As we are always seeking to improve our products, the information in this document gives only a general indication of the product capacity, performance and suitability, none of which shall form part of any contract. We reserve the right to make design changes without notice. All trademarks are acknowledged. © Cobham 2015.

The most important thing we build is trust

For further information please contact:

Cobham Wireless

Ireland: +353-1-236-7002

USA: +1 408-385-7630

[email protected]

the most important thing we build is trust - testforce.com · the most important thing we build is...

Documents