fast packet processing in the cloud

Slide title

CAPITALS

Slide subtitle

minimum 30 pt

Fast packet processing in the cloud

Dániel Géhberger Ericsson Research

Slide title

Text and bullet level 1

minimum 24 pt

Bullets level 2-5

minimum 20 pt

Characters for Embedded font: !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™ĀĀĂĂĄĄĆĆĊĊČČĎĎĐĐĒĒĖĖĘĘĚĚĞĞĠĠĢĢĪĪĮĮİĶĶĹĹĻĻĽĽŃŃŅŅŇŇŌŌŐŐŔŔŖŖŘŘŚŚŞŞŢŢŤŤŪŪŮŮŰŰŲŲŴŴŶŶŹŹŻŻȘș−≤≥fifl

ΆΈΉΊΌΎΏΐΑΒΓΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΪΫΆΈΉΊΰαβγδεζηθικλνξορςΣΤΥΦΧΨΩΪΫΌΎΏ

ЁЂЃЄЅІЇЈЉЊЋЌЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯЁЂЃЄЅІЇЈЉЊЋЌЎЏѢѢѲѲѴѴҐҐәǽẀẁẂẃẄẅỲỳ№

Do not add objects or text in the

footer area

Ericsson Internal | 2016-04-14 | Page 2

› Motivation

› Service chains

› Hardware related topics, acceleration

› Virtualization basics

› Software performance and acceleration

› Packet Processing Execution Environment

› Conclusion

Outline

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› High bandwidth

– Up to 10Gbps but a few 100Mbps should be generally available

› Low latency

– 1 ms 5G radio latency

– Optimal network deployment for low end-to-end latency

– Remote control use-cases

› High spectrum (NX) radio technology, massive MIMO, beamforming

– https://www.youtube.com/watch?v=Sqcv_a9GGUk

› Network slicing

– Create separated networks in seconds

› Fully cloud based deployment from the antenna site

– Network Function Virtualization and service chaining

› Support for IoT (the smart hype)

Motivation? 5G!

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Splitting up legacy network components

– Core network examples: per user charging, rate limitation

– Radio example: PDCP functionality

› IPSec termination and GTP de-tunneling

› Sequence numbering, buffering

› Header compression

› Air-crypto and transmission towards the antenna site

› Possibly per user service chains

› Dynamic configuration

› 3rd party components

› Optimization for performance

– Hardware capabilities and acceleration, software technologies

– Placement: physical location, noisy neighbors

Service chains

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Throughput: 10/25/40/100 Gbps, dual port

› Interconnect

– (RJ45+CAT7 for 10Gbps) / SFP+ / QSFP+

– Latency!

› PCIe counts

– PCIe v2, 16 lanes: 64Gbps

– PCIe v3, 16 lanes: 128Gbps

› Offloading

– Checksum calculation

– TCP/UDP reassembling

– Tunneling

– RDMA over Ethernet

– Receive Side Scaling (RSS)

Modern Ethernet NICS

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› 1 CPU core can handle ~10Gbps in the best (and simple) case

› Distribute the incoming packets between cores

› RSS hash: IP addresses, transport type, transport ports

– Assign the packet to an RX queue (descriptor ring)

– Default: # of queues = # of CPU cores, interrupts

› Sometimes it cannot work: encrypted IPSec flow

› We have TX queues as well

› SR-IOV uses these as well

NIC Queues

NIC Hash

RX queue 1

RX queue 2

RX queue n

CPU core 1

CPU core 2

CPU core n

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› More than one CPU sockets

› Dedicated memory and PCIe

– Slower access from other sockets

– Where to schedule the packet processing tasks?

– Use the local memory

NON-Uniform memory access NUMA

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› “Clouds will be built with off-the-shelf hardware elements”

› Partially true, but…

– Amazon, Google, Facebook and the other big players do not really follow this

– They can ask Intel for custom designs and build their own servers

› Hardware acceleration

– (GPU)

– Cryptography

– Packet re-ordering

– Using ARM and Network Processing Units instead of Intel

– Smart NICs

Commodity hardware?

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Open vSwitch data path in the NIC

Smart NIC example Netronome

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Intel – Ericsson joint datacenter hw/sw development

› Today exists only in a limited version, research topic

› Separated sleds for compute, networking, storage, (memory)

– Optical backplane interconnect

– Smart NICs

› Software defined hardware

– “Give me physical computer with 3 cores, 5GB memory, and 1TB disk”

– CPU slicing

– New type of OS?

– Latencies?

– Virtualization?

Rack Scale architecture Hyperscale datacenter

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Linux container – Common kernel

– 1 application

› Docker – Packaging!

› Data to container

Virtualization Technologies

› Library OS – Lightweight kernel

› Only kernel space

– Focus on 1 application

› Can be a JVM

– Well-known hypervisors again!

› Standard VM – Well-known hypervisors

› KVM, Xen, ESX, etc.

– Full kernel

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

Hypervisors

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

Packet forwarding with OVS 2.0

• Figure: mega packet per second!

• 14.88 millions of 64 byte packets per

second on 10G interface

• A packet arrives every ~67.2 ns

• 3.5 GHz 1 cycle = 0.28 ns

• 240 cycles for everyting for line-rate

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Simple ping RTT measurement with ~10000 samples

› Not tuned for low latency

Latency

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Kernel overhead – System calls

– Context switching on blocking I/O

– Data copying from kernel to user space

– Interrupt handling in kernel

Default packet processing with Linux

User space

Kernel space

Driver

queues

Socket Ring

buffers

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› Fast packet processing solution from Intel

› Not only for Intel NICs any more

– Other vendors, virtual interfaces

› Set of libraries for applications

– Switch, VNF, etc.

› Polling packets

– Uses a dedicated CPU for 100%

› Direct user-space access

› Lockless buffer rings

› Batch packet handling

› Huge memory pages

› Pre-allocated buffers

DPDK Intel Data Plane Development Kit

User space

Kernel space

App DPDK Ring

buffers

UIO driver

queues

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

OVS 2.0 with DPDK and KVM

› The price:

– DPDK has to support the NIC

– DPDK aware switch, hypervisor and

application

– 3 CPU cores with a VM

› 1 in the switch for NIC VM

› 1 in the VM

› 1 in the switch for VM NIC

› (So the figure is not completely fair)

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

Power consumption

0 500 1000 1500

Packet size

BananaPI (ARM) Atom Xeon

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› ER Flow Switch

– 12 Mpps for packet forwarding through a VM or container

– JIT linking

› Packet Processing Execution Environment

– JIT linking, custom callbacks in the switch

› By extending OpenFlow

– Heavy PPs in VMs/containers

– Specific PPs on NPU, ARM, etc.

switching to Ericsson Research

Slide title

minimum 24 pt

Bullets level 2-5

minimum 20 pt

footer area

› 5G networks will be cloud based

– High bandwidth, low latency

› Acceleration is required

– NIC extra features

– Dedicated hardware acceleration

– User space fast packet processing

Conclusion

fast packet processing in the cloud

Documents

dpdk packet processing ia overview presentation

gaspp:&agpu(accelerated& stateful& packet&processing

ffpf: fairly fast packet filters

chapter 6 packet processing functions

a fast and scalable ipv6 packet classification

fast userspace packet processing - uliege · fast userspace...

load scheduling for flow-based packet processing on...

fist: a fast, lightweight, fpga-friendly packet latency...

efﬁcient techniques for fast packet...

fast packet processing with kubevirt · 2020. 8. 31. ·...

storage performance development kit overview · fd.io –...

stateful inter-packet signal processing for wireless

dpdk & layer 4 packet processing

high-performance packet processing engines using...

fast packet processing using ebpf and...

parallelizing packet processing in container overlay...

x86 hardware for packet processing

received packet processing at the dev layer...

tuplemerge: fast software packet ... - nonsns.github.io

private money-cred-packet-fast track pwp