nfv testing & devops | qualitest
TRANSCRIPT
NFV Testing & DevOps
Agenda
NFV – Challenges NFV Testing & DevOps Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
2
Agenda
NFV – Challenges NFV Testing & DevOps Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
3
4
Bryon LowenVice President Telco at QualiTest
Benny Sand NFV Strategy Lead at QualiTest
Webinar Presenters
About QualiTest Group
| World’s 2nd Largest Independent Testing Company
| QualiTest was recognized as the only visionary pure play QA and testing Service Company in
| the “ Magic Quadrant”
5
Partial list of Telecom Customers
6
Agenda
NFV – Challenges NFV Testing & DevOps Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
7
NFV Predications
| NVP planning phase is shifting gear
| Operators are moving along the NFV curve
| Financial forecasts indicate a significant growth in NFV investments
| NFV will start to influence budgets and purchasing decisions
| NFV adaption pace is progressing faster then expected
8
NFV impacts on OPEX
| OPEX Redactions
| Shared computing resources
| Reduced power, space, cooling requirements
| Executing trials of new services, without incurring much risk
| Proof of Concepts (POCs) and trials can be run faster
| Dynamic recovery from failures
| Accelerated time -to-market
| Stronger capability to bring up services anywhere in the world at any time.
9
NFV impacts on CAPEX
| The use of commodity servers reduces hardware costs
| No dependence on specialized hardware
| A single, common server architecture can be used within the data center environment.
| Migration to a distributed cloud architecture allows operators to deploy backup in a cost effective way
| Cycling the hardware more often improves the overall performance of the network
10
Strategic challenges I
| NFV involves very complex integration of various software elements
| Everything known becomes unknown
| Network testing expertise versus Virtualization and Software testing
| Agile orientation will be introduced in Network testing
11
Strategic challenges II
| Variety of NFV infrastructures means that more testing will be required by the carriers
| Frequent updates in the Network services will require more management and orchestration testing
| More test automation is required to support agility, impact testing and regression testing
| NFV test strategies are needed to accelerate and ensure the delivery of NFV solutions with guaranteed quality
12
Agenda
NFV – Challenges NFV Testing & DevOps
Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
13
What is DevOps?
| Who: Developers, QA and Operations| What (Goal): Clarity, Consistency and
Collaboration to understand the changes that each release brings to the IT and customer environments| Why: Increase business benefits by reducing the
transaction cost associated with delivering incremental change. | How: Methodology and tools that continue to
evolve
14
Where does testing fit in DevOps?
| Addressing a misconception| Testing is not a bottleneck| Testing is a junction point between
development and operations| Testing is a change agent
| DevOps is about closing the distance between development and operations through:| Meaningful collaboration| Early engagement| Continuous processes
15
Shift Left Principle and implementations
| Engage QA early| Push tests to lower levels| Test continuously| Benefits:| Fewer defects created| Defects discovered earlier| Cost per defect drastically reduced
16
Why Network DevOps is Key to SDN and NFV
“There remains much to do before this vision [Domain 2.0] can be implemented, including pivots from networking craft to software engineering, and from carrier operations models to cloud “DevOps” models. We also see an important pivot to embrace agile development in preference to existing waterfall models.
”
17- AT&T’s Domain 2.0 initiative white paper
Positioning QA with Development and Operations
| Development
| Engage as early as possible in the life cycle
| Generate test automation at lower levels within the product
| Reduce the overall demand for manual testing and testing at the GUI level
| Operations
| Testing and Monitoring in all environments• Environments should be as
production-like as possible• QA needs more control and access
to environments| Continuous feedback and
communication| Testing of Operations processes
18
Agenda
NFV – Challenges NFV Testing & DevOps
Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
19
Building SDN / NFV Network Dev Ops assurance process
20
Build strong assurance process to allow Network DevOps to capitalize SDN / NFV potential
| Renegotiate contracts
| Right Price
| Good SLAs
| Increasingly Complex
| Renegotiate contracts
| Not affect QOS
QualiTest NFV validation process
QualiTest aims to create a unified end-to-end approach to service assurance and network performance
21
NFV Management Orchestration Infrastructure Migration
The Evolution of the Modern Tester
| Traditional Tester Skills
| Analytical and Logical Thinking| Intellectual Curiosity| Critical Thinking and Rational
Deduction| Ability to Identify and Apply
Fundamental Knowledge
| Modern Tester Skills
| Analytical and Logical Thinking| Reading and Writing Code| Grasping the Bigger Picture| Ability to Recognize and Address
Design Challenges| Ability to Communicate on both
Technical and Business Needs| Ability to identify appropriate use
cases for manual testing (for example, exploratory testing for learning and test design)
22
Agenda
NFV – Challenges NFV Testing & DevOps
Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
23
NFV framework – Test areas
Overall, the NFV framework consists of three main components:
1.Virtualized Network Functions (VNF)
2.NFV Infrastructure (NFVI)
3.Network Functions Virtualization Management and Orchestration Architectural Framework (NFV-MANO Architectural Framework)
24
Example:MMS VNF Test Strategy
Multimedia Messaging Service (MMS) is similar to the Short Message Service (SMS) in that a message is passed between an Originating Party (OP) and a Destination Party or Parties (DP).
The core Network Element is the Multimedia Messaging Service Centre (MMSC) which acts as a central store and forward server between the OP and DP, as shown below.
25
MMS VNF Network Dev Ops Test Approach
| Static testing (drive defect curve)
| Scope (requirements/functionality of VNF)
| Shape test (risk based / ISO9126)
| Define levels of testing
| Create Test environment (NFV-TE):| Simulation tools| Network analytics| QoS / QoE probes| Onboarding Automated Certification Process
26
QualiTest NFV validation process
Assure full compliance to ETSI ISG Standards.
•Define NFV Testing process•Test Requirements•NFV Test Environment•Risk Assessment•Roles and Responsibilities•Team Skills and Training•Test Inputs / Basis
Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions.
Test Environment: simulation tools, network analytics, QoS and QoE probes.Automated Certification Process for each VNF before it is introduced to the network
Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM).
Test the VNF architecture and implementation to assure smooth transition.
NFV Test Policy
ETSI NFV ISG Compliance
(NFV-Co)
Selecting Mano Vendors
(NFV-Ven)
NFV Staging Environment
(NFV-SE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
27
28
The Need for Automation| DevOps produces Up to x30 Code
Shipments
| Completing deployments up to 8,000 times faster
| Automation becomes a must have
| Coverage to reach 100%
29
Traditional vs. DevOpsAutomation Element Traditional DevOps
What to automate? Mainly sanity and regression New features
Automation Trigger Manual CI tool
Staff Automation experts All-around with deep business knowledge
Tools Commercial Open-source
Automation Owner QA Dev + Ops + QA
Automation techniques GUI + API API
Dependency between scripts Flow Independent automation units
Where to automate Test Environment Dev + Test + Production environments
DevOps Automation
30
| Every layer of testing should be automated
| The concept of executing all regression test cases for a test pass is soon becoming obsolete.
| Strategy around testing new features needs to be formalized and interim builds can be supplied to QA
Test strategy goals
| Build an NFV Ecosystem
| Drive Quality and governance
| Continuous integration / Dev Ops model
| Reduce cost of integration
31
Agenda
NFV – Challenges NFV Testing & DevOps
Enablement
Summary & Questions Network DevOps Test Approaches Network DevOps
Introduction
32
Summary
| NFV allows agile real time shifting of assets to where they are needed
| Testing, in the new world of NFV, changes dramatically and becomes a core discipline that is central to the overall goal of pleasing customers in a cost effective way
| NFV testing is the change agent from the physical networks to the virtualized ones
| DevOps leveraging On-Demand Environments is the industry direction
| Adapting new technologies like On-Demand Environments with Test Automation will enable your organization to MAXIMIZE REVENUE
| DevOps improves time to NFV solution implementation| Network DevOps is the future in NFV
33
34www.QualiTestGroup.com/NFV
www.QualiTestGroup.com