Addressing 5G Network Function Requirements

Tieto benchmarks QoS and IPSec* gains with Intel® FPGAs and Intel® PAC

Introduction5G is now an area of significant strategic focus in the communications industry. With deployment starting in 2019, there are potentially transformational shifts to look forward to:

• 5G NR will introduce new radio technology, using directional antennas, higher order MIMO, and millimeter wave (mmWave) signals to deliver higher bandwidth and streaming performance

• New 5G hardware will bring gains in energy efficiency while using the radio spectrum more efficiently

• New capabilities, such as the ability to deliver multiple service levels over the same infrastructure, will enable new business models

• Applications including remote driving, MMC sensors, and mission-critical communications will be possible at scale

• 5G platforms will also deliver additional security capabilities and new metrics/statistics for self-optimization and automation

Today, use of NFV technology goes hand in hand with the development of 5G. Efficient use of more-flexible, high-performance solutions based on Intel® Xeon® server technology will be essential to realize the promise of 5G. NFV and container technologies are key to enabling cloud-style deployment and programmability (scaling, upgrades, multicloud, etc.) for legacy custom hardware in fixed-function radios. Increasing flexibility relies on providing scalable systems that reach peak performance for demanding next-generation 5G services when needed. Thus, a solution that balances the flexibility of a programmable cloud platform with the attributes of a programmable hardware solution, such as an FPGA, is critical to achieving these goals.

This white paper explores how the new 5G requirements can be handled using a Programmable Acceleration Card (PAC) based on Intel® technology.

Two different benchmarks were used to explore how IPSec* and QoS requirements for 5G networks can be addressed. The benchmarks were conducted to show how an efficient, high-capacity, low-latency solution can be built to augment the general-purpose CPU for these applications on the host Intel Xeon processor.

Intel® FPGAs and Intel® PAC5G QoS and IPSec Benchmarking

Increasing performance and flexibility using an Intel®

FPGA-enabled Programmable Acceleration Card for 40 Gbps

and 5G networking for IPSec* and QoS

Table of Contents

Introduction . . . . . . . . . . . . . . . . . . . . 1

5G Network Traffic Use Case Requirements . . . . . . . . . . . . . . 2

Intel PAC Solution . . . . . . . . . . . . . . . 2

Benchmarking Process . . . . . . . . . . . 3

Benchmarking QoS . . . . . . . . . . . . . . 4

Benchmarking IPSec Offload . . . . . 7

Summary and Findings . . . . . . . . . . 8

white paper

White Paper | Addressing 5G Network Function Requirements

Enhanced mobile broadbandEnable mobility and throughput

Critical machine type communication (cMTC)Improve latency, reliability, and availability

Smartphones Home, enterprise

4K/8K UHD, virtual reality/AR, broadcasting

Massive machine type communication (mMTC)Control device cost, volume, and energy consumption

Smart building and city Smart agriculture

Smart meters Capillary networks

Logistics, tracking, fleet management

Public safetyImprove integrity, reliability, and availability

Warning Blue light Governmental

Traffic safety and control

Industrial applicationsRemote manufacturing,

training, surgery Emergency

KG

112

5G Network Traffic Use Case RequirementsGiven the wider scope of 5G, the requirements for eMBB, mMTC, and URLLC use cases will be broad. Building key elements of the network using Intel® server technology, combined with a PAC, provides the flexibility to address the complexities of these use cases.

Intel PAC Solution5G use cases introduce new requirements for performance, security, robustness, reliability, and end user experiences. The 5G network is dimensioned for the overall traffic load in the serving area, rather than for the accumulated peak rate of all cells. Increased traffic load in the network will drive the demand for acceleration to efficiently manage use of server capabilities. The Intel PAC technology can enhance system capabilities—even at 40 Gbe—enabling additional flexibility for system scaling.

Even when traffic volume is handled, there are additional compute workloads (functions) that respond well to offload.

Figure 1 . 5G Network requirements

5G eMBB and mMTC use caseThe eMBB and mMTC 5G use cases will drive increased overall traffic (packet per second, bits per second) in the networks enabled by new technologies. The eMBB use cases drive the possible peak rate per device. The mMTC devices generate a low amount of traffic but, due to a huge number of devices, a large amount of accumulated data.

The new technologies enable increased bandwidth per cell, which will put higher demands on IPSec acceleration capabilities on the cell level.

The higher frequency bands for the air interface drive the densification of the radio network and, thereby, the increased number of cells.

5G URLLC use caseThe URLLC 5G use case will drive the need to minimize the end-to-end latency. This includes both mean latency and the tail of the latency distribution.

The end-to-end latency includes the latency on the air interface as well as latency through the network. The latency through the network includes internal latency in the user-plane nodes and latency between the nodes. The traffic carried by the network will be a mix of flows with different QoS requirements. Differentiation of services is needed to be able to prioritize the low-latency packets.

The tail of the latency distribution of a reliable communication protocol is sensitive to dropped packets (packet loss ratio).

Figure 2 . Intel® PAC

Intel® Arria® 10 FPGA Accelerated function IP and gearbox

Intel® Ethernet Controller XL710 Intel® Fortville Ethernet NIC

10/40 GbE

QSFP-A 4x 10G or4x 25G QSFP-28

QSFP-8 4x 10G

2

White Paper | Addressing 5G Network Function Requirements

Each additional function adds complexity and latency to a system, and for 5G, latency is key for many use cases. By accelerating workloads that can benefit, such as IPSec and QoS, optimizing overall server utilization provides a basis for supporting additional workloads and requirements for 5G and mobile edge computing (MEC), as well as highly functional service chains and microservice architectures.

A great way to minimize latency is to minimize the number of secure sandboxes that traffic needs to go through and thereby the number of IPSec tunnel/de-tunnel steps in the end-to-end service chain. Today, this is limited within networks and data centers. But with increasing demands on multiple workloads on distributed clouds, the requirements are increasing for secure communication between, and increasingly within, networks and data centers and even for processing elements in a distributed solution.

In these complex environments, integrating end-to-end service chains on an Intel PAC can greatly reduce user-plane latency and remove nondeterministic behaviors while increasing the overall capabilities of the system to handle these complex workloads. Compared to a fixed offloading function, an Intel PAC also brings the benefit of supporting multiple packet processing functions that can be colocated and run in parallel, increasing overall system flexibility.

Benefits of using Intel® PAC • Fully programmable solution

• Augments the system with a flexible offloading solution, increasing total system capabilities

• Leverages offloading of certain workloads to free up system resources for additional workloads

• Offers a more deterministic solution for total system performance and latency

Benchmarking ProcessThe test results in this white paper were made using the Intel PAC board (codenamed Vista Creek).

In order to analyze the benefit of offloading, two important traffic scenarios supporting 5G requirements were benchmarked: high throughput QoS and IPSec traffic. An Intel PAC solution with the Intel Xeon processor and high-performance interfaces based on Intel’s Data Plane Developer Kit (DPDK) was used to improve overall system performance and flexibility. A fully customizable pipeline in the Intel PAC offloads key workloads and enables system resources to be utilized in an optimized way.

QoS benchmarking setup, as shown in Figure 3, was done in a “black-box” device under test (DUT) setup with tester and DUT running on a single Intel® Xeon® Scalable processor Linux* server using DPDK and PROX/DATS with a 40 Gbe networking interface.

For IPSec, three different setups were used, benchmarking both single-server and dual-server setups with 40 Gbe networking interfaces. The test framework is PROX/DATS-modified to support IPSec functionality.

Figure 3 . Benchmark system setup for QoS

Figure 4 . Benchmarking system setup for IPSec*

Intel® Server Board S2600WF

DATS

PROXloadgen

PROXreceiver

PROXCLI

CPU/Socket 1

Configuration throughshared memory

Configuration offeature through,

e.g., DPDK TM API

Packet processingpipeline

PROXlooper

CPU/Socket 0

PCIe* bus

NAP-board(FPGA)

40G

PCIe* bus

40G

FVL/XL710

PROXOffloadIPSec

Intel®FLY

board

Intel®FLY

boardNIC 2

Intel®MAP

boardNIC 1

SWEnc/Dec

PROXSW

IPSec

Port 0

Fortville

Port 0

Port 1 Port 1

HWEnc/Dec

3

White Paper | Addressing 5G Network Function Requirements

Benchmarking test setupSoftware environment baseline• OS version: CentOS* Linux release 7.4.1708 (core)

• Kernel version: 4.14.15-1.el7.elrepo.x86_64

• DPDK v17.11.0

• PROX v0.41

DPDK QoS benchmark• System under test CPU cores running a trivial DPDK

• PMD forwarding device application, including queue mapping, in cases of egress traffic management (egress TM) offload

DPDK IPSec benchmark• System under test CPU cores running full software IPSec

DPDK PMD application, in cases of non-IPSec offload

• System under test CPU cores running an IPSec in-line pre- and postprocessing DPDK PMD application, in case of IPSec offload

Hardware bill of materials (BOM)PCIe* plug-in Intel PAC with one of the following FPGA RTL IPs:• Hierarchical egress TM IP, featuring queue management,

strict priority and weighted scheduling, and traffic shaping - Up to 128K queues (2 GB memory), 64K shapers, and 1x40 Gbps ports per board

• IPSec accelerator IP, featuring in-line IPSec acceleration - IPv4/ESP/transport mode with up to 8K Rx SAs, 8K Tx SAs, AES-GCM 128/256-bit keys, 2x25 Gbps ports per board

Both IPSec and QoS were benchmarked on the Intel PAC with functionality offloaded to the Intel Arria 10 FPGA. For the benchmarking, 40 Gbe of traffic was loaded onto the system, looping in the Intel Xeon processor host cores to showcase total system performance at 40 Gbe line rates.

This minimized load on the host systems for improved performance and decreased latency, as well as increased functionality and flexibility.

Figure 5 . 5G service data flows overview (QoS IPSec*)

Traffic flow template

UPF (5GC)

Best effort(web)

Video(YouTube)

Generic VoIP(Skype)

Video(Netflix)

Video

Video

Control

IMS VoIP(VoLTE)

QoS flow 1

QoS flow 2

QoS flow 3

QoS flow 4

QoS flow 5

QoS flow 6

QoS flow 7

QoS

flow

mar

king

IPSecDeciph.

IPSecCiph.

(optional)mappingto DSCP

QFIinsertion

GTP-Utunneling

Benchmarking QoSToday, the Intel PAC accelerated QoS solution is successfully deployed to handle the demanding nature of broadband network gateways (BNGs) and edge routing functions. Many of these advanced features will also be seen throughout the 5G network.

End-to-end QoS and a high-quality end-user experience are key advantages of the 5G network. A flexible solution, supporting end-to-end throughput and QoS, will help ensure the demanding requirements of 5G are met. The solution benchmarked supports multiple 5G use cases to meet end-to-end network requirements—with scalable throughput and an ability to handle scenarios where the granularity of QoE ranges from a few to several thousands of classes with very fine granularity, and per user traffic of up to 128K separate queues per board deployed.

5G standardized 5QI specifies traffic classes for guaranteed bit rate (GBR), non-GBR, and delay-critical GBR with a packed delay budget as low as 10 milliseconds for non-delay critical and acceptable packet error rates ranging from 10-2 to 106.

The system was tested for a range of packet sizes and queues, from a few hundred up to several thousand queues,

to explore the different end-to-end QoS requirements of a 5G network. Looking at end-to-end QoS, a solution to handle the wide range of requirements through the network with deterministic behavior is key. Benchmarks done on the Intel PAC show how requirements can be handled in a flexible manner with high traffic throughout, while supporting the required packet error rate and latency parameters.

4

White Paper | Addressing 5G Network Function Requirements

QoS functions were benchmarked to analyze behavior and system dimensioning with both 25 Gbps and 40 Gbps shaping using different traffic mixes over 1024 up to 128K QoS queues. A packet-size sweep benchmark was also done to analyze the total system performance and core usage without the QoS functions. No software-only solution was found that could meet these requirements. Initial comparisons show that using the Intel PAC reduces the required host CPU processing power 300 percent at 25 Gpbs (versus running these workloads with software only). For 40 Gpbs, only the Intel PAC-enabled system was tested.

Figure 6 shows that, for the Intel PAC solution for most traffic mixes, a single core can handle up to 25 Gbps and two cores are sufficient for 40 Gbps.

Using 25 Gbp and 40 Gpb shaping, a system can handle line rate except for very small packages using a single core for 25 Gbps and two cores for 40 Gbps. This system dimensioning was used for the benchmarks in this white paper.

In Figure 7, detailed analysis of QoS functions shows how a system with 25 Gps shaping handles overall set shaping requirements and prioritization between two queues (where TC0 = high-priority traffic, TC1 = low-priority traffic). The performance was measured to find the highest load the system could handle with less than 10-3 maximum accepted packet loss.

Figure 6 . Base packet sweep system dimensioning test

Avg. latency TC0 (us)Throughput TC0 (Gbit/s)Avg. latency TC1 (us)Throughput TC1 (Gbit/s)

HW QoS Characteristics, Ethernet packet size 512 bytes

Thro

ughp

ut [G

bit/

s]

Design load [Gbit/s]

Late

ncy

[us]

800000

700000

600000

500000

400000

300000

200000

100000

00 5 10 15 20 25 30 35 40

20.0

17.5

15.0

12.5

10.0

7.5

5.0

2.5

0

Loss (%)Throughput (Gbit/s) Core load (%)

HW QoS Characteristics, Ethernet packet size 512 bytes

Thro

ughp

ut [G

bit/

s]

Design load [Gbit/s]

Core

load

and

loss

[%]

100

80

60

40

20

00 5 10 15 20 25 30 35 40

25

20

15

10

0

5

Throughput Core usage

Thro

ughp

ut [G

bit/

s]

Packet size [bytes]

Core

usa

ge [c

ores

]1.4

1.2

1.0

0.8

0.6

0.4

0.2

00 250 500 750 1000 1250 1500 1750

40

35

30

25

20

10

15

Throughput Core usage

Thro

ughp

ut [G

bit/

s]

Packet size [bytes]

Core

usa

ge [c

ores

]

1.4

1.2

1.0

0.8

0.6

0.4

0.2

00 250 500 750 1000 1250 1500 1750

40

35

30

25

20

10

15

Figure 7 . QoS function analysis

5

White Paper | Addressing 5G Network Function Requirements

A detailed analysis of how the system scales up to 128K user queues shows the benefit of the high-memory throughput onboard memory of the Intel PAC solution.

25G shaping1 core

1K queues

64K queues

128K queues

Throughput 25.06 Gbps 24.89 Gbps 24.92 Gbps

Total core usage 0,33 cores 0,33 cores 0,33 cores

25G shaping1 core

1K queues

64K queues

128K queues

Throughput 39.99 Gbps 39.70 Gbps 39.58 Gbps

Total core usage 0,64 cores 0,64 cores 0,64 cores

Total system latency was measured for 40 Gbps shaping and 1K queues. Average latency was 150us round-trip (with a minimum of 29us and maximum of 1 millisecond).

Benchmarks for QoS show MAP-board egress traffic manager individual functions, like queue management, strict priority scheduling, and packet rate shaping, are accurate when scaled to maximum configuration capacity and operate at loads up to 40 Gbps Ethernet without accuracy degradation.

CPU loads are fully independent of egress TM functionality configuration and operation when offloaded to the Intel PAC, once each packet is tagged with a queue ID.

In Figure 8, a detailed analysis of latency, CPU core load, and packet loss minimum latency was assessed—all factors key for 5G use cases. The system performs with reduced load

on the host system CPUs with latency in the submillisecond range. The minimum latency measured through the system was ~20us.

Figure 9 shows that the system scales from 1K to 128K queues up to close-to line rate performance without reaching set packet loss values.

The benchmarking performed shows that the Intel PAC QoS solution egress TM individual functions, like queue management, strict priority scheduling, and packet rate shaping, are accurate when scaled to maximum configuration capacity, and operating at loads up to 40 Gbps Ethernet without accuracy degradation. CPU loads are fully independent of egress TM functionality configuration and operation when offloaded to Intel PAC once each packet is tagged with a queue ID.

Figure 9 . Scaling number of queues

Figure 8 . Design load sweep detailed analysis, 40 Gpbs with 1K queues and 2 cores

System load (% of line speed)Latency avg (us)

Design load sweep

Core load avg (%) Pkt loss ratio (x10-6)

100

80

60

40

20

04 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38

128k Queues1k Queues 64k Queues

KPI — QoS DPDK Offload

Thro

ughp

ut [G

bit/

s]

Load [Gbit/s]0 5 10 15 20 25 30 35 40

40

35

30

25

20

10

5

0

15

40

38

36

34

32

30403836343230

6

White Paper | Addressing 5G Network Function Requirements

Figure 11 . Comparison graph offload IPSec* at 48 Gpbs with 4.8x less load on the system with acceleration

Figure 12 . Detailed analysis of IPSec* running only in the software or with the Intel® PAC at different packet sizes

No IPSec

IPSec* vs. software IPSecIPSec offload SW IPSec

Thro

ughp

ut [G

bps]

CPU

cor

e co

unt

60

50

40

30

20

10

0

12

10

8

6

4

2

0Throughput

47.9 Gbps48.0 Gbps48.0 Gbps

Core count

9.5 cores

4.6 cores

3.3 cores

IPSec* comparison packet sweepCore usage, offloaded (cores) Core usage, no offload (cores)

50

40

30

20

10

0128 256 512 1024 1500 5000 9000

9

8

7

5

3

4

6

2

1

0

Acceleration benefitBase system load Host system cores IPSec*

10987

5

34

6

210

Accelerated Without acceleration

Figure 10 . Running 48 Gbps IPSec* traffic

Benchmarking IPSec OffloadTo explore the benefit of PAC acceleration, IPSec is chosen as a compute-intensive workload. IPSec is one of the protocols used to provide security for today’s networks and, as discussed, requirements for security and capacity increase for 5G networks.

Figure 10 shows that both software-only and Intel PAC scenarios are capable of running 48 Gpbs of traffic with IPSec. For comparison, the base system load for packet processing is included; this shows the use of the Intel PAC provides an acceleration factor of 4.8x for the host CPU load used for IPSec processing. The results were based on an IPSec packet size of 578 B, IPSec key size of 256 bits, core configuration of 6-6, and 48 G traffic generated, with CPU cores running a full software IPSec DPDK PMD application for a non-IPSec offload, and an in-line pre- and postprocessing DPDK PMD application for IPSec offload.

Comparing these results, the benefit of adding the Intel PAC is shown in Figure 11, where the system is using significantly fewer of the host system CPU cores.

Figure 12 shows the same configuration, with a detailed comparison of how the IPSec function scales based on different packet sizes from 128 bytes to 9,000 bytes. Even for very large packages, the IPSec processing takes significantly more processing power at 3.5 cores compared to Intel PAC acceleration, where only 0.2 core is needed to process IPSec in the host CPU.

7

White Paper | Addressing 5G Network Function Requirements

Tests measure performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit intel.com/benchmarks.

Performance results are based on testing as of October 2018, and may not reflect all publicly available security updates. See configuration disclosure for details. No product can be absolutely secure.

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information about benchmarks and performance test results, go to intel.com/benchmarks.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer, or learn more at intel.com/fpga.

Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.

Intel, the Intel logo, Arria, and Xeon are trademarks of Intel Corporation or its subsidiaries in the US and/or other countries.

*Other names and brands may be claimed as the property of others.

© Intel Corporation

0219/BH/CMD/PDF 338687-001US

QoS and IPSec* Offloading for 5G Network TrafficThis white paper covered two key aspects of 5G requirements: security and end-to-end QoS. The benchmarks conducted show that an NFV-type environment using Intel® Xeon® processors to address the flexibility and performance needs of 5G can greatly benefit from an Intel® PAC to offload demanding compute workloads, while boosting overall system performance and latency.

QoS benchmarks show that we can handle complex QoS requirements with a single Intel PAC board with 40 Gbps and thousands of queues, reducing latency and packet loss. With 5G requirements pushing end-to-end system QoS down to 10 milliseconds, reducing latency at every node and processing element in the network will be important. IPSec benchmarks show that compute-intensive workloads can benefit from an Intel PAC, with both the software-based solution and the Intel PAC capable of processing 48 Gbps of IPSec traffic. The addition of the Intel PAC reduces core usage by nearly 5x.

Summary and FindingsToday, software-defined networking (SDN) and network function virtualization (NFV) are about trading performance for programmability and visibility to enable a multitude of new services. With 5G, this becomes a larger constraint as bandwidth and latency is critical and we see that an Intel PAC can be leveraged to augment server CPUs to balance these requirements.

The need for programmability and insight for advanced 5G and networking use cases means that more code and functions will be added to packet processing, and this will require more compute power, which can be augmented by an Intel PAC running packet processing on Intel FPGA-based accelerators. Added benefits include:

• Reduced reliance on general-purpose cores, enabling them to be reserved for high-value use cases like edge computing, analytics, or closed loop optimization.

• The ability to manage QoS, shaping, and advanced traffic optimization with limited impact on latency and CPU processing needs, and improving the performance of the overall system.

• Traditionally, a trade-off between performance and security has been required in high-speed packet processing scenarios. Functions like IPSec can be offloaded to the Intel PAC enabling the overall system to handle complex workloads for high-throughput scenarios.

• Each processing step and transition (decryptions, shaping, etc.) adds latency and limits the total throughput— with an Intel PAC these issues can be mitigated and processed in a more deterministic way.

The Intel PAC can be explored further for service functions chaining (SFC) and more advanced analytics where multiple IPSec encryption tunnels and handovers can yield great benefits. This white paper has explored host-terminated scenarios; however, some traffic classifications could be added to the Intel PAC to create a fast path for different traffic types where only part of the traffic flow is used for higher-value processing at the CPU cores.

With distributed cloud and MEC—which will require even more SDN and NFV application flexibility with improved 5G performance and latency—there are further benefits of using a flexible, scalable server-based environment assisted by an Intel PAC for certain workloads. The capability to accurately plan and distribute workloads means more efficient utilization of infrastructure and faster turnaround for evolving customer needs in future networks.

Learn MoreFor more information about Intel FPGAs and Intel PAC, visit intel.com/fpga.

Find out more about 5G at intel.com/5G.

Read an article on 5G network transformation.

For more information about Tieto, visit tieto.com.

8