Micro Focus Network Virtualization: The Secret Sauce to Awesome ApplicationsUnderstanding the Impact of Network Conditions and Applying Them in Your Testing

Network Virtualization

eBook

Table of Contents

Network Application Performance—No Guessing Allowed 4

How It Works 5

Accurate Network Conditions Mean Accurate Testing 6

When to Use Network Virtualization Software 7

Can’t I Just Test Across Our Real Network? 8

Don’t Load Generators Do the Same Thing? 8

Using Network Virtualization Software for Web Application Testing 9

What About In-Browser Analytics? 9

Don’t Content Distribution Networks Solve Web Performance Problems? 10

Can’t I Impose Network Loads with Fiddler and Other Proxies? 10

Using Network Virtualization for Mobile App Testing 11

Can’t I Test Mobile Apps on the Real Network? 12

Won’t New Cellular Technologies Such As 4G and 5G Help? 12

Using Network Virtualization for Infrastructure Changes 13

Enabling Nearly Continuous Integration and Testing 14

Realize the Benefits Now 15

Learn More 15

Appendix 1—Obtaining Network Conditions 16

How the Network Capture Server Operates 16

Reporting and Network Profiles 18

Operation of the Network Capture Express Mobile App 18

Operation of the Global Library 19

Appendix 2—Replaying Network Conditions 20

How Network Virtualization Software Works 20

Network Virtualization Software Accuracy 21

Network-Layer vs. Application-Layer Network Impairment 21

Why Network Virtualization Is Better than Bandwidth Throttling Alone 22

Using LoadRunner Virtual User Generator (VuGen) 22

Overhead on Load Test Infrastructure 23

Overhead on a Workstation or Server 23

Virtual Appliance Mode 24

Performance and LoadRunner Protocols Supported 24

Testing a Website Using a Content Distribution Network (CDN) 25

Deploying Network Virtualization Software for Mobile On-Device Testing 26

Deploying Network Virtualization Software and Mobile Center 27

Appendix 3—Reporting and Analysis 28

Analysis and Reporting Options 28

eBook

Network Application Performance—No Guessing AllowedHow might your application perform when you place it in production? What would happen if you move a critical web app into the cloud? Or consolidate servers in a new data center? Would users find your new mobile app useful and productive, or would they simply uninstall it? The answer to all these questions is… It depends.

It depends on many things. But in the age of connectivity, one of the things it depends on is the network perfor-mance. Excessive network latency or packet loss doesn’t just make your app respond slower; it can also cause the application to behave erratically or even fail outright. About 40 percent of applications we have tested are sensitive to the impact of the network—their behavior and scalability change as users encounter different network conditions. So testing network impact on applications during development, and prior to infrastructure changes, is crucial. And fine-tuning the application to perform well in the expected network conditions helps enable its success.

Unfortunately, it’s not as easy as it used to be. Network complexity is staggering, and user loads are unpredictable. Com-munication often occurs over the Internet from remote users, mobile apps, and virtual private networks, so the makeup of the network connection itself is often unpredictable.

Micro Focus® Network Virtualization software is a performance testing solution that lets you accurately test the impact of the network on applications during the development process, or before you implement major infrastruc-ture changes—even in highly complex network configurations. And, it provides the data you need to improve the application’s performance.

This eBook explains what Network Virtualization software does, how it works, and how you can use it to deploy better performing, more usable applications.

eBook

How It WorksThe major challenge to accurate network impact testing is to represent the expected network configuration and load—and to integrate the results with application performance testing tools. Wide-area network (WAN) emulation software or appliances have historically been used to emulate network conditions in various environments. However, while these solutions might be useful for testing the capacity of a network, they are complex to use when testing the network impact on applications. They often don’t accurately represent the performance parameters of the real network, and they don’t integrate well with application testing solutions, so complexity is high and return on investment is low.

Network Virtualization software, on the other hand, works by sampling real network conditions from the actual net-work. The resulting network characterization—although not the traffic itself—is then accurately “replayed” during performance or functional testing to impose the expected network impairment on the running application.

You can use Network Virtualization software to reproduce the impact of either client-to-server or server-to-server com-munications. It emulates real network behavior including latency, packet loss, bandwidth limitation, and other behav-iors of real networks. It lets you analyze the traffic flowing during tests, so you can determine the reasons for slowed network performance. And it provides recommendations for optimizing the application.

Network Virtualization software is available for Windows and Linux. It can be installed standalone, but it’s closely integrated with Micro Focus Performance Center solutions such as LoadRunner, Mobile Center for mobile app testing, Sprinter manual testing solution, and StormRunner Load software as a service. There is no need to change scripts or load infrastructure configurations.

eBook

Accurate Network Conditions Mean Accurate TestingThe network often slows the performance of remote users, and in some cases it causes failures due to retries or packet loss. Small changes in the underlying net-work latency can have large impacts for some applications. For example, packet loss can severely impact operating system and application behavior by resetting the TCP window during data transfer. Bandwidth and the impact of queuing, reordering, or duplicating packets also affect performance.

Using accurate network performance conditions—replayed in the network layer—is cru-cial to discovering the actual application behavior prior to deployment. In some cases, dynamic network changes and jitter can impact the ability of the application to recover. Static profiles do not typically induce these failures; however, the dynamic network per-formance profiles enabled by Network Virtualization software enable you to discover the impact before deployment.

The Network Capture server available in the Network Virtualization suite is a highly flex-ible agentless network monitor that lets you simply and quickly record, import, and re-play production network conditions, such as available bandwidth, latency, and packet loss. The Network Capture mobile app—available for iOS and Android devices—records network conditions on mobile devices and allows you to email the recorded profile to use it in performance testing. Another bundled component, the Global Library, provides sample network conditions for up to 4,000 global locations, covering mobile, WAN, and Wi-Fi users.

Network Virtualization software also includes a network profile editor that lets you manually describe network conditions using randomized profiles, or import recorded profiles from any source. Simple changes to the profile—changing from 3G to 4G wireless technology, for example—let you compare performance differences between the two alternatives. You can also generate manual profiles as simple XML files and import them into the editor to model very specific scenarios, such as periods of dis-connection or network congestion.

eBook

When to Use Network Virtualization SoftwareYou can use Network Virtualization software to test any kind of application—for exam-ple, web apps, enterprise applications, or mobile apps—that are delivered across any kind of network, including: LANs, WANs, remote office networks, home networks, and mobile and satellite networks. Recorded network conditions can be applied during both functional and non-functional (performance and load) testing. It can also be used on any platform, including servers, desktops, and mobile devices. Network Virtualization is key when validating mobile web and mobile apps, because performance may vary con-siderably due to network conditions.

You can also use Network Virtualization software to apply network conditions be-tween application tiers in the backend to model the impact of infrastructure changes. For example, you can predict the impact of data center migrations or relocations to prevent surprises and reduce the risk associated with re-hosting.

You should use Network Virtualization software during the development process when performing both functional testing and load/performance testing. Even when local functional testing shows that the application performs the functions specified, net-work-induced timeouts might cause remote users to experience login issues, be un-able to complete a search, or submit a transaction, because the network slows their communication to the server. Including the accurate replay of network conditions during functional testing enables you to detect and fix remote experience problems prior to deployment.

Optimization analysis provided within Network Virtualization software can also be a crit-ical metric during development cycles to validate and maintain the quality and perfor-mance for all users. Network Virtualization analysis should be integrated into the test and deployment processes to enable issues to be detected prior to release.

eBook

Can’t I Just Test Across Our Real Network?Yes, you can. But Network Virtualization is better. Here’s why:

� Real networks carry live traffic that can produce false test results.

� Testing across real networks could impact production systems, especially if you perform load or performance tests.

� You can’t accurately repeat test scenarios. The live network can change between executions, so you can’t determine if you have fixed observed issues.

� You can’t test “What-if?” scenarios that don’t exist in the real network.

� Executing tests remotely can be complex and require additional hardware and resources.

In many cases, the real network isn’t available prior to application deployment.

Network Virtualization software completely removes the requirement to test across the network, while adding high-fidelity network emulation to uncover problems. You can record network conditions in the live environment—so you have a faithful repre-sentation of the network—and replay them accurately in the test environment. This allows you to repeat tests with the same network performance conditions to validate that you have resolved issues.

Don’t Load Generators Do the Same Thing?Remote load generators (load injectors) are sometimes used to replicate the load of us-ers from specific remote locations (remote offices, call centers, data centers) across the live network. They can provide useful information, but they can also introduce problems:

� Testing can affect production systems or must be done after hours.

� Production network issues or congestion can influence the test results.

� The cost of maintaining remote load generators is significant.

� What-if? scenarios cannot be tested—only the existing locations can be tested.

� You can’t easily model web and mobile connected users.

Load generators hosted in the cloud or an external network raise new problems. Al-though they might have a network profile that differs from those hosted inside the network, this can often be a well-connected, high-bandwidth network that does a poor job of representing the actual end users.

Network Virtualization software allows all network conditions to be emulated without mov-ing the load generator, including the ability to replay multiple conditions per load generator, and to replay individual web and mobile conditions for each simulated user separately.

When cloud or externally hosted load generators are used, you still need a network performance solution such as Network Virtualization software to simulate the realistic network conditions users might experience.

eBook

Using Network Virtualization Software for Web Application TestingNetwork Virtualization software enables you to apply accurate network performance con-ditions to web application tests to uncover performance issues prior to deployment. Network Virtualization software operates at the network layer below the operating system and browser, so it enables download performance and resulting browser behav-ior to be identical to a remote user even for complex deployments. Network Virtualiza-tion software even enables you to replay multiple conditions concurrently, with differ-ent content virtually appearing in different remote hosted locations on the same test.

Network Virtualization software provides extensive analysis of traffic flowing during tests to offer automated optimization recommendations to improve performance. You can also use transaction markers for analysis and reporting, including step timings for manual tests or integration with any automation framework to support replay of exist-ing or new scenarios.

What About In-Browser Analytics?Compared to in-browser analytics such as FireBug or YSlow, Network Virtualization software provides a complete network validation process, including sampling and profil-ing the real network conditions, replaying them during testing, and reporting for analysis and optimization. Similar to Firebug and YSlow, Network Virtualization software reports the un-optimized use of HTTP and HTTPS elements to deliver content, but it adds ad-ditional analysis features. FireBug and YSlow are also limited to specific browsers only. Network Virtualization software analytics operates on the packet data flowing in the network, and therefore supports all applications that use HTTP or HTTPS, including desktop apps, desktop web, mobile native, hybrid, and web apps.

Network Virtualization software analytics also provide a range of unique reporting and enhancement capabilities, including throughput graphs, element filtering and sorting, mobile and web image size analysis, and many more. Because Network Virtualization software applies network conditions to the tests, it enables you to compare application performance in impaired and unimpaired situations.

eBook

Don’t Content Distribution Networks Solve Web Performance Problems?Content distribution networks (CDNs), such as Akamai, Limelight, Riverbed, Velocix, Edgecast, and Amazon Cloudfront, improve performance for remote users by cach-ing content in the Internet to reduce long-haul latency and ease the load on the original servers. This technique is effective for improving performance of high-volume global websites, but there are gaps that can leave your apps vulnerable to perfor-mance problems. For example:

� They often can’t cache dynamic content—only static elements that do not change.

� They don’t address the performance impact of the last mile connection (apart from compressing content not already compressed), so web users on poor connections might not see much benefit.

� They don’t work well for small audiences spread over large geographic areas, because caches don’t get populated due to low numbers of requests.

� They require careful content and hosting configurations—small errors can result in large problems.

� They are potentially expensive to deploy.

� They are only suitable for external audiences connecting via the Internet, not those using a VPN connection.

If you use a CDN for deployment, you should also use Network Virtualization soft-ware to validate the remote performance for users, to enhance the application for com-munication across the last mile, and to test the scalability and performance of the ap-plication servers.

Can’t I Impose Network Loads with Fiddler and Other Proxies?Proxies such as Fiddler, MITM, and Charles Proxy manage HTTP and HTTPS traffic flowing between clients and servers. Some of them provide bolt-on mechanisms to throt-tle or impair traffic. But these proxy-based network impairment solutions are typical-ly implemented in the application layer. Because of this, the processes collecting the measurements have their own effect on network performance, which can result in inaccurate measurements.

Network Virtualization software operates in the network layer. This provides greater ac-curacy because the vast majority of solutions today are at the application level and not at the driver level. Other reasons Network Virtualization software is a better solution include:

� It can be used with any IP traffic, not just HTTP and HTTPS.

� It supports all network impacts, including latency, packet loss, jitter, and bandwidth.

� Network Virtualization software enables you to record and replay actual network conditions.

� It enables you to apply dynamic network profiles and complex network scenarios, including multiple conditions simultaneously.

� It provides the Global Library, allowing sample conditions to be discovered.

� Network Virtualization software offers detailed reporting and analytics, including enhancement recommendations.

Proxies are still useful in many situations. You can implement them alongside Net-work Virtualization software to redirect client HTTP and HTTPS traffic through a server where the Network Virtualization software test manager is operating, without the need to modify the network configuration. Clients then simply need the proxy to be configured to allow Network Virtualization software to add network conditions. A proxy is also a good way to decrypt HTTPS traffic without the server-side key.

eBook

Using Network Virtualization for Mobile App TestingThe network impact on mobile application performance is especially critical for busi-ness success. A recent study1 of more than 3,000 mobile users in the U.S. and Europe demonstrates how mobile performance can impact revenue, brand reputation, and customer loyalty. Some highlights:

� 76% of respondents judged app performance as critical or very important.

� 49% expect a mobile app to respond in less than two seconds.

� 55% hold the mobile app responsible for poor performance, not the network or device.

� 73% say mobile app performance or reliability issues caused them to have a lower opinion of the app provider.

Mobile on-device performance testing is often focused on verifying that the app meets the functional requirements. But when deployed, users can use the app under all possible mobile and Wi-Fi network conditions, which can severely impact perfor-mance and behavior. Even content delivery systems for mobile don’t help performance in the critical last mile, which is where many mobile problems occur. Functional testing with only local Wi-Fi, or a specific real mobile network in the test lab location, can often leave slow performance, functional failures, or aberrant behavior undetected until deployment.

For comprehensive mobile app testing, devices can be connected using Wi-Fi, and Net-work Virtualization software used to accurately replay network conditions to simu-late mobile networks. This enables you to have an excellent chance of detecting and fixing application performance problems prior to deployment.

__________

1 TechBeacon, “2015 mobile app usage and abandonment survey, April 2015. slideshare.net/TechBeacon/2015-mobile-app-usage-and-abandonment-survey

eBook

Can’t I Test Mobile Apps on the Real Network?Not easily and not accurately. Attempting to use actual mobile networks for testing adds extra complexity due to the need to manage SIMs and mobile contracts. And testing multiple devices using a single mobile network cell in one location limits the results to a single-connection scenario, and could cause overloading that affects test results.

Using Network Virtualization software to virtualize network conditions is better than using real networks, because it enables you to accurately model network conditions and replay them repeatedly. That way you know any changes in application perfor-mance or behavior result from changes you have made to the app, not changes in network conditions. More complex scenarios can also be modeled, including discon-nects, network changes, and others. Network Virtualization software also provides many other benefits, including recording and replaying conditions, analysis, reporting, and optimization.

Won’t New Cellular Technologies Such As 4G and 5G Help?New mobile network technologies can improve performance—for some users in some circumstances. However, they won’t reduce the need to test using Network Virtu-alization. New technologies seldom reach 100% of the population, even when fully deployed, due to the cost and complexity of deployment. Even when widely available, there is a lag because consumers do not replace devices immediately or universally.

The new technologies sometimes use higher frequencies to deliver more data. Higher fre-quency transmissions are more subject to interference, so they don’t penetrate buildings as well. They might also suffer impairment from other sources, such as metal infrastructure or even the human body. When users move indoors, travel in vehicles, or move behind ob-structions, devices frequently have to drop down to lower frequency transmissions (3.5G, 3G, or even 2.5G) to remain connected, and performance might be impaired.

Finally, cell capacity is a shared resource. Even when you have a strong signal, your effective throughput can be reduced if you connect to a busy cell site. This is why it is often impossible to download data in crowded locations and why good network throughput experienced when a technology is first released can deteriorate as net-work usage grows.

eBook

Using Network Virtualization for Infrastructure ChangesInfrastructure changes—data center moves, consolidations, or migration of apps or serv-ers into the cloud—bring excessive risk. Our experience has shown 40 percent of appli-cations might be impacted by data center change and at least 10 percent might suffer severe degradation.

Network Virtualization software enables you to test and assess the impact of data cen-ter change to application performance, functionality, and scalability during the planning phase. It shows you which business processes expect to suffer degradation. It pro-vides recommendations for how to remediate applications. And it helps you determine whether WAN acceleration or other technologies might help.

You can use Network Virtualization software in the test environment in different ways to support testing different scenarios. For example:

� On workstations for client-side testing, to determine the impact to end users of changes in server host location

� Alongside Micro Focus Performance Center or Micro Focus LoadRunner for load testing, to determine the scalability impact of multiple end users when server location changes

� On servers or in a virtual appliance mode (between client and server or between servers), to test the back-end impact of changes in server location

In virtual appliance mode, Network Virtualization software is deployed on a dedicated physical or virtual Linux server through which traffic is routed. Network Virtualization soft-ware can then impair traffic flowing in the network during test activities.

eBook

Enabling Nearly Continuous Integration and TestingAgile development processes, nearly continuous integration and testing, and DevOps are driving business agility by enabling organizations to put new application functionality in users’ hands in days rather than months. Network Virtualization software fits cleanly into today’s Agile and DevOps development and release cycles, so you can be confident your applications can perform as expected when deployed.

The Network Virtualization Network Capture server and Network Capture mobile app bring accurate, real-world network conditions into your nearly continuous test-

ing environment. Full integration into our extensive range of functional, performance, and load test tools enable accurate, scalable testing during development cycles. Also, RESTful APIs enable integration into many test automation frameworks, such as JMeter, Selenium, Appium, LeanFT, Gatling, and others. APIs also let you access Network Vir-tualization software configuration functions, stop and start testing, control the network conditions applied during the test, retrieve test parameters, and access and control other aspects of the test.

When integrated in this way, Network Virtualization software reports and perfor-mance indicators of actual end user experience verify conformance with service level agreements—before you deploy the app.

eBook

Realize the Benefits NowThe network can slow down your applications—typically from 2 to 1,000 times the local response. So if you deploy web, mobile, or other applications across LANs, the Internet, VPNs, private networks, or mobile networks, Network Virtualization soft-ware shows you the impact in advance. You reduce the risk of failed deployments, customer dissatisfaction, and loss of revenue from web sales or brand damage.

Unlike other network performance solutions, Network Virtualization software sam-ples real conditions from your real network—latency, packet loss, jitter, and others—and accurately applies them during application tests to give you a realistic view of how the app can perform in production. Out-of-the-box integration with Micro Focus Per-formance Center, LoadRunner, StormRunner Load, Unified Functional Testing, and Mobile Center, alongside open APIs to support all other test frameworks, makes adding Network Virtualization to your functional and load testing programs easy.

A customer survey2 found that customers using Network Virtualization software re-duced application performance incidents by an average of 18 per year, and saved $1.08 million annually in remediation costs. And, internal costs can pale in comparison to loss of sales and damage to the brand that can result from failed applications.

Learn MoreGet started with Network Virtualization software today. Learn more at software.microfocus.com/software/network-virtualization.

__________

2 Shunra Software, “Shunra for HP Customer Survey Results,” July 2011

eBook

Appendix 1—Obtaining Network ConditionsHow the Network Capture Server OperatesNetwork Capture enables you to record network conditions from clients or servers to other targets. It is suitable for recording and profiling conditions between data centers, remote offices, or remote workers, so you can replay them with Network Virtualization software for testing. The network conditions typically cover client to server connec-tions for applications such as web, SAP, and virtual desktop infrastructure, as well as server-to-server environments.

Network Capture is accessed remotely from a browser.

Architecture and Components

� Network Capture Server—Web service and repository for data. Each server can support up to 25 concurrent measurements and can be installed in the Internet or inside a private network.

� Network Capture agents—Installed on Windows workstations and servers. Each agent can support up to 20 concurrent measurements to remote targets, which can be anything connected on a network that can respond to the requests sent to it, including servers, clients, web servers, routers, and firewalls. No software is required to be installed on the remote target. Agent installation on remote target is required only for measuring bandwidth for both directions.

� Measurements and Monitors—Each measurement can be configured to run for periods from one minute to one month, from a Network Capture agent to a remote target, which can be agentless or have an agent installed. Measurements record latency and packet loss at a specified frequency, usually every three to five seconds, and can capture bandwidth estimates every five minutes. For each measurement, you can configure one or more specific protocols, including ICMP, TCP, UDP, or HTTP.

� Network Profiles—Selected Network Capture results can be exported for replay in the Network Virtualization software.

eBook

Measurement Types Latency and packet loss measurements can be setup to run:

� From a Network Capture agent to an agentless remote target with no software installed. Measurements can use ICMP (ping) or TCP to a specified port, which might require a service on the remote workstation to respond to the TCP request.

� From a Network Capture agent to a web server using an HTTP HEAD request.

� Between two Network Capture agents. These measurements can be set up using UDP on a specific port in addition to the other measurement types.

Bandwidth measurements can be setup to run:

� From a Network Capture agent to an agentless remote target. This measurement uses a mix of UDP and ICMP packets and can provide a one-way bandwidth estimate sending several packets.

� Between two Network Capture agents. This measurement can support two-way bandwidth estimates sending a small amount of data using a mix of UDP and TCP packets, or a two-way destructive bandwidth test, which exhausts the available bandwidth.

Other NotesNetwork address translation (NAT) can be supported on a remote target, provided the target responds to the request and the protocol and request is allowed.

Firewalls block specific communications and might disallow requests in specific di-rections. You might need to select protocols and ports that are open in the firewall, or add exceptions to firewall rules to allow specific Network Capture measurements to operate.

Network Capture measurements across a virtual private network (VPN) connection are possible, but you might need to select a protocol that can be routed via the VPN.

The network impact of using Network Capture is simply related to what measurements and protocols are set up, at what frequency, from which agents, and to which targets.

eBook

Reporting and Network ProfilesOnce measurement results have been created in the Network Capture server, you can inspect them using the web interface. You select the time period to be reported on and the reports generated for the selected timeframes, which include percentile distributions and summary statistics.

Search tools are also available to locate the best or worst network measurements, based on latency or estimated bandwidth. You can export selected periods of time as a net-work profile in an .ntx file for direct import to Network Virtualization software. And you can save profiles to the server, so they can be reused by others.

Operation of the Network Capture Express Mobile AppThe Network Virtualization software Network Capture mobile app is available free of charge for both iOS and Android platforms from the appropriate app store. It allows you to test latency, packet loss, and bandwidth on a mobile device, and then share the pro-file for use in Network Virtualization software tests. It requires you to enter a remote server, which should respond to ICMP ping. Otherwise, the latency and loss values might be empty—100% loss. You can configure the app to test for between 1 and 15 minutes, and it can cycle during the test repeatedly, testing latency and resulting packet loss, and download bandwidth.

The app sends ICMP ping requests to the remote server to measure latency and loss. Download bandwidth measurements use servers hosted in the Internet to download data to the mobile device. This does not involve the selected remote server, but tests the last mile connectivity to identify the bandwidth limits of the connected network. We measure the bandwidth from a server on the Internet, because the mobile net-work bottleneck is usually located in the last mile between the mobile device and the cellular provider.

You can email the test results as an .ntx file, which has a simple XML format. You can then replay it in the Network Virtualization software or enter the test results manually when defining conditions.

eBook

Operation of the Global LibraryThe Global Library allows users to specify a server and client location to obtain sam-ple network conditions for mobile and broadband networks. The data in the library is syndicated from several third parties and comprises the results of end-user network tests executed globally in desktop browsers and on tablets and smartphones. The raw data comprises on hundreds of millions end-user tests, covering more than 1,000 client and server locations, and categorized based on the type of network connection and other characteristics.

The Global Library selection process initially uses the distance between the select-ed client and server locations to calculate the network latency due to distance. Then it combines this with the selected client network connectivity to provide a represen-tative network profile.

Here’s how the process works.

You select the client network characteristics based on the data available for the client location, including technology, mobile carrier, and business hours.

The Wi-Fi technology type represents users of desktops, tablets, and smartphones that ran network tests but were connected to a Wi-Fi network. The throughput of a Wi-Fi network to the Internet is usually not dictated by the Wi-Fi network itself, but by the home or office Internet connection behind the Wi-Fi network. So the download band-width results vary by individual test and cannot be correlated. If the Wi-Fi technology type is selected, then bandwidth might not be specified within the profile and would be defined as “unlimited.” However, you can manually modify it when replaying. For mobile technology types, latency, loss, and bandwidth values are provided.

In the final step, you select poor, fair, or good communication quality, which corre-sponds to the 5th, 50th, and 95th percentile of the available data. You must use cau-tion when selecting the poor or good conditions because they can vary considerably due to statistical variation of the input data. You should compare poor or good net-work conditions against other locations and confirm the expected network conditions as representative.

eBook

Appendix 2—Replaying Network ConditionsHow Network Virtualization Software WorksIn Network Virtualization software, network impairment is implemented using a network driver installed beneath the operating system in the TCP stack. Traffic flowing into and out of the selected interface passes through the Net-work Virtualization software driver where the required network modeling is applied. This is intrinsically more accurate than alternative network impairment solutions that operate at the application level.

For the standalone version of Network Virtualization software, a companion Test Manager is also installed along-side the driver to provide a web and mobile user interface and other functionality. An optional Network Virtual-ization software server can also be installed to support user authentication and file storage, and to act as a floating license manager.

When Network Virtualization software is installed alongside Performance Center or LoadRunner software, different Network Virtualization software components are installed on the Performance Center server, the controllers, and load generators. And the Network Virtualization software user interface is embedded within the Performance Center or LoadRunner Controller.

eBook

Network Virtualization Software AccuracyWe have tested Network Virtualization software in multiple configurations to determine the accuracy of the network modelling and the overhead on the test systems. These two parameters are related because an overloaded system can result in variability in network impairments. Consequently, the test environment must not be overloaded prior to adding Network Virtualization software.

The CPU and memory impact of adding Network Virtualization software is in the range of 0 to 25 percent, with a typical impact of less than 5 percent. We have validated the network impairment to be within one ms of the request-ed latency for 99.9 percent of requests, provided the system is not overloaded and a supported system configura-tion is used.

We have also validated Network Virtualization software in high throughput scenarios and have verified that it operates within the tolerances stated above. However, you should note that the addition of latency or loss into a network might reduce the effective throughput, both theoretically and actually. This is not a limitation of Network Virtual-ization software, but the expected impact of delaying packets flowing through the network. Adding latency might reduce the available throughput of a network; for example, a one-Gbps network connection would typically sup-port considerably lower throughput than one Gbps if latency is added.

Network-Layer vs. Application-Layer Network ImpairmentNetwork-layer impairment—as implemented in Network Virtualization software—is implemented in a device driver below the operating system. This means that the operating system or application under test cannot detect that anything is changed from the normal operation of the network, and it can behave exactly as if the network con-ditions were real. This is important because the objective of virtualizing networks is to induce failures that can occur after deployment.

In application-layer impairment, network modeling is implemented as part of another application—usually a test-ing solution or proxy. This type of impairment is inherently inaccurate, because it interferes with the flow of data between client and server in a way that can be detected, potentially changing the behavior of the application and invalidating test results.

eBook

Why Network Virtualization Is Better than Bandwidth Throttling AloneBandwidth throttling is the practice of controlling only the data flowing into and out of applications in an attempt to model real-world conditions. Some software solutions provide this as a network test feature (for example, within the run-time settings in LoadRunner VuGen script replay). That’s not Network Virtualization, however, and it doesn’t enable the same accurate model as Network Virtualization software.

Many networks are limited by bandwidth. But using bandwidth throttling alone may impact activities that down-load enough data to reach the limit, while all other traffic may flow unimpaired, providing an extremely inaccurate model. All traffic is affected by latency and loss. That’s what frequently causes application problems and slowdowns, and it is far more important to model latency and loss. Bandwidth is only relevant when it is low enough to impact the application—but usually is an addition to latency and loss.

Network Virtualization software supports the complete, accurate modeling of all aspects of networks, including latency, packet loss, jitter, and bandwidth.

Using LoadRunner Virtual User Generator (VuGen)The network speed simulation in LoadRunner VuGen has been superseded by Network Virtualization software, which provides much more accurate test results.

VuGen runtime settings support “Network Speed Simulation” specified as a single bandwidth value. This technol-ogy is implemented in the application layer, making it potentially inaccurate, because the behavior of packets that exceed the limit cannot be correctly modelled. In addition, it supports only a single combined bandwidth for upload and download.

In many cases, the main impact of the network is due to latency. Every packet is affected, but only those packets ex-ceeding the bandwidth limit can be impacted by a bandwidth-only model. So steps that download a lot of data can potentially be throttled, but other traffic is unaffected.

eBook

Overhead on Load Test InfrastructureNetwork Virtualization software operates on the load generator, running network em-ulation in software within the network layer. This means that there might be a small overhead to using Network Virtualization software when running load tests. This can vary between a limited increase in CPU and memory requirements to, in some cases, a lower CPU load with a higher memory requirement. We have tested Network Virtualiza-tion software extensively within LoadRunner and Performance Center software to verify that the overhead is within acceptable limits.

The exact impact of Network Virtualization software primarily depends on the network conditions applied. For example, high latency or low bandwidth profiles can slow the cycle time of virtual users, resulting in reduced CPU load. But if the script transfers large vol-umes of data at the same time, it might require more memory to store the network traffic in the virtual network.

Alternatively, a low latency or high bandwidth profile might only slow the throughput of the virtual user a small amount, resulting in a small increment in CPU load and limited additional memory requirements.

In many cases—provided the load generator operates within acceptable limits with-out Network Virtualization software and non-extreme network profiles have been con-figured—the impact might be limited and not need a change in the number or type of load generators required for tests.

Overhead on a Workstation or ServerOn a workstation or server, Network Virtualization software runs network emulation in software within the network layer. This results in some overhead that can vary be-tween a small increase in CPU and memory requirements to, in some cases, a lower CPU load with a higher memory requirement. We have tested Network Virtualization software extensively to determine that the overhead is within acceptable limits.

As with the load test infrastructure, the exact impact of Network Virtualization soft-ware depends on the network conditions applied. For example, a high latency or low bandwidth profile can slow application performance, resulting in reduced CPU load. But if the test transfers large volumes of data, it might require more memory to store the network traffic within the virtual network. On the other hand, a low latency or high bandwidth profile might only slow the throughput of the test a small amount, resulting in a small increment in CPU load and limited additional memory requirements.

In many cases, provided the test environment operates within acceptable limits with-out Network Virtualization software and provided non-extreme network profiles have been configured, the impact might be very limited—in the range of 0 to 25 percent with a typical impact of less than 5 percent. This should not require any changes in the client or server configuration.

eBook

Virtual Appliance ModeIn virtual appliance mode, you deploy Network Virtualization software on a dedicated physical server and route traffic through the server. You can then use Network Virtualization software to impair traffic flowing in the network to support network virtualization.

When deployed in a virtual appliance mode, multiple NICs can be used with traffic passed between NICs using routing (standard Linux kernel routing). Or it can be deployed in a “bridge,” where traffic can be mirrored between multiple NICs concurrently with Network Virtualization software, applying impairments to all traffic. You can also use VLAN trunking.

You can access the Test Manager web interface remotely to manage the network impairments. You can also support multiple concurrent users managing tests and multiple network conditions per test. You can control network traffic by IP address range or even protocol and port range, with different conditions supported in the same test. The interface provides dynamic throughput reporting and transaction marking, in addition to optional packet capture and extensive analysis and reporting capabilities, either within the Test Manager interface or standalone analytics tool.

Performance and LoadRunner Protocols SupportedNetwork Virtualization software supports all protocols. Some protocols (Citrix ICA, COM/DCOM, Java Record/Reply, Java vuser, MAPI) are supported using the “per load generator” setting. In that case, network conditions are applied on the whole load generator, and all virtual users share a single network condition. All the remaining protocols are supported in both “per group” and “per load gen-erator” settings. The “per load generator” setting does not support applying individual bandwidth per virtual user.

The older Network Virtualization software—v8.6 for Performance Center and LoadRunner versions 12.00 to 12.20—support all protocols except Citrix ICA, COM/DCOM, Java Record/Reply, Java vuser, and MAPI. TruClient and SAP GUI protocol support with Network Virtualization software v8.6 was introduced in LoadRunner and Performance Center 12.00, so for older LoadRunner and Perfor-mance Center versions between 11.52 and 12.00, these protocols are not supported.

eBook

Testing a Website Using a Content Distribution Network (CDN)Testing a website that can be deployed using a CDN presents extra challenges. CDN technology can improve performance overall. However, it can result in a wider variance of end-user experiences, be-cause some users might use cached content available locally, whereas other users might have to pop-ulate the cache (requesting the content from the original servers). CDNs are also used to reduce the number of external users served from the original servers. But small changes in content, misconfigura-tion, or user activity can cause huge increases in “pass-through” traffic allowed to flow through the CDN caches, resulting in scalability and performance problems on the original servers.

It is more complex to determine the end-user experience for remote users before deployment, be-cause the external CDN cannot be used until content has been externally hosted and configured. And when considering origin server scalability in CDNs, applying load using standard tools might not match the actual load seen once users connect via the external CDN. Additionally, many CDN providers forbid load testing through the live network.

Network Virtualization software provides capabilities to virtualize the network conditions for end us-ers, and it can also be used to prototype performance via CDNs. There are several approaches to replicate CDN behavior prior to deployment, including hosting content on multiple local servers and using Network Virtualization software to apply different conditions to each. You can also use a local proxy with caching.

For scalability testing of web or mobile applications using CDNs, the recommended approach is to consider that 30 percent of the overall traffic might pass through to the origin servers. The load pro-file might also need to reflect the effect of caching and the impact of HTTP 304 request and response cycles between the CDN cache and origin servers. As stated above, scalability testing is actually more critical for CDN-accelerated content, because the original servers are usually not sized to support the full user community (relying on the CDN to take the majority of the load). That means small miscon-figurations or changes can result in huge changes of pass-through load.

eBook

Deploying Network Virtualization Software for Mobile On-Device TestingFor mobile testing, you can deploy Network Virtualization software alongside a proxy to simplify adding network impairments to the mobile test devices. The mobile test de-vices can be connected on a local Wi-Fi network rather than using a mobile connection.

The proxy is used to redirect mobile HTTP and HTTPS traffic through the server where the Test Manager is operating. Mobile devices simply need the proxy to be configured to allow Network Virtualization software to add network conditions.

eBook

Deploying Network Virtualization Software and Mobile CenterMobile Center includes licenses for Network Virtualization software. Mobile Cen-ter provides simple interfaces for defining the network conditions in Unified Functional Testing software, Sprinter software, and the VuGen component of LoadRunner. And it manages the API calls to Network Virtualization software to verify that the conditions are started for the specified device.

Network Virtualization software is deployed alongside Mobile Center on a dedicated Windows server. A proxy is used to support connections from mobile devices, which needs to be configured on the devices. (This is not done by Mobile Center.)

The Mobile Center server sends API requests to the Network Virtualization software server and the mobile test devices connect to the Network Virtualization software serv-er to send traffic via a proxy. This might require the mobile devices to be connected on Wi-Fi with the Network Virtualization software server also connected in the same net-work. The individual IP addresses of the mobile devices must be visible to the Network Virtualization software server.

You must apply the proxy configuration to the mobile devices, install the proxy local certificate on the devices, and add proxy IP addresses and ports in the network set-tings. The Mobile Center server must be configured with the IP address of the Net-work Virtualization software server in the /opt/hpmc/conf/server.properties file.

Additional contact information and office locations: microfocus.com/about/contact

eBook

Appendix 3—Reporting and AnalysisAnalysis and Reporting OptionsThe Network Virtualization software Test Manager provides a web interface you can access from almost anywhere in a web browser. It includes many features for analysis and optimization.

Learn More Atsoftware.microfocus.com/software/network-virtualization

335-000014-001 | 4AA6-5788 | H | 12/17 | © 2017 Micro Focus. All rights reserved. Micro Focus and the Micro Focus logo, among others, are trademarks or registered trademarks of Micro Focus or its subsidiaries or affiliated companies in the United Kingdom, United States and other countries. All other marks are the property of their respective owners.