nvfi: l2 or l3
TRANSCRIPT
V2.1 | ©6WIND 2015. All rights reserved. All brand names, trademarks and copyright information cited in this presentation shall remain the property of its registered owners.
SPEED MATTERS
V2.1 | 2 ©6WIND 2015
ETSI NFV Model : Openstack the defacto VIM
V2.1 | 3 ©6WIND 2015
ETSI NFV Model
Neutron : APIs and Python agents to configure hypervisor’s
networking
V2.1 | 4 ©6WIND 2015
Requirements of Virtual Infrastructure
OpenStack: analysis of a Linux KVM NFVI case
Hypervisor
Virtual Switch
Driver Level - pNICs
L2 Virtual Switch or
L3 Virtual Routing
Virtual NICs (vNIC - virtio) - Host to Guest
Virtual Machine (KVM/Qemu)
Virtual
Machine
Application
Software
Virtual
Machine
Application Software
Server Platform
VM-to-VM Communication (ETSI Phase2)
NFV
Infrastructure
VNFs
V2.1 | 5 ©6WIND 2015
ovs-appctl
ovs-vsctl
ovs-dpctl
ovs-ofctl
+ what if OpenDayLight is being used?
NFVI survival kit: learn OVS debugging
V2.1 | 6 ©6WIND 2015
pNICs = 10G/40G PCI Ethernet ports
vNICs = virtio
Neutron leverages:
L2
OVS
Linux Bridge
L3
netns / Multi Tenants
iptables
Firewall -> Security Groups
NAT -> Openstack’s floating IP addresses
L3 routes
VPN
Kernel IPsec (XFRM)
Overlay / tunneling
OVS vPort or Netdevices
VXLAN, GRE
(GENEVE, MPLS)
Neutron : Linux’s consumer of pNICs to vNICs technologies
dnsmasq
VM 1
eth0
tapYY…
qbrXX
qvbXX
br-int
qvoXX
qr-yyy
br-ex
eth0
qg-yyy
br-tun
eth1
patch-int
patch-tun
External Network
virtio / vhost device
Linux Bridge
veth Pair
OVS
Physical Interface
Network Namespace
OpenStack compute node (host)
tapzzz
IP
IP
IP
IP
qrouter-aaaa
qdhcp-bbbb
Virtio vNIC
pNIC
Qemu / KVM
V2.1 | 7 ©6WIND 2015
Limited Bandwidth To
Linux Based Virtual
Machines
Test 1:
Linux Open vSwitch and Linux VM
X86 server with
12x10G Ports
Hypervisor
Linux Based
Virtual Machine
Open
vSwitch
IP
Forwarding 14
Gbps
V2.1 | 8 ©6WIND 2015
Compute
Node 240Gbps 6WIND Virtual Accelerator
throughput on 12 cores of Xeon E5-2697 v2
@ 2.70GHz
1 core provides a 20Gbps Virtual Accelerator
bandwidth
Examples on a dual socket / 24 cores server
120Gbps North-South traffic delivered to standard
VMs or VNFs with 12 cores remaining for VMs
40Gbps North-South traffic with 20 cores remaining
for VMs
40Gbps North-South traffic and 160Gbps East-
West traffic for service chaining
6WIND Virtual Accelerator in OpenStack Compute Node
Standard VM/VNF
(DPDK, Linux,
other OS)
Virtio
Standard VM/VNF
(DPDK, Linux,
other OS)
Virtio
Virtual Accelerator
Host
Drivers Virtio
Physical
NICs
Virtual Switching
Open
vSwitch
Linux
Bridge
Overlays
GRE
VXLAN
VLAN
Virtual Networking and
Multi-tenancy
IP Fwd
VRF Filtering
NAT LAG
IPsec
V2.1 | 9 ©6WIND 2015
Compute Nodes Neutron Diagram
dnsmasq
VM 1
eth0
tapYY…
qbrXX
qvbXX
br-int
qvoXX
qr-yyy
br-ex
eth0
qg-yyy
br-tun
eth1
patch-int
patch-tun
External Network
virtio / vhost device
Linux Bridge
veth Pair
OVS
Physical Interface
Network Namespace
OpenStack compute node (host)
tapzzz
IP
IP
IP
IP
qrouter-aaaa
qdhcp-bbbb
6WINDGate
6WINDGate
fast path
PMD PMD
PMD
V2.1 | 10 ©6WIND 2015
Test 2:
6WIND Virtual Accelerator + Linux VM
8X Bandwidth
Increase
X86 server with
12x10G Ports
Hypervisor
Linux Based
Virtual Machine
Open
vSwitch
IP
Forwarding
14 Gbps
118
Gbps
Virtual Accelerator
V2.1 | 11 ©6WIND 2015
6WINDGate OVS L2 switching performance
6.8 Mfps per core
Up to 67.8 Mfps using 10 cores (20 threads)
Performance scales linearly with the number
of cores configured to run the fast path.
Performance is independent of frame size.
This benchmark is simplified, other
overhead have to be included:
Host to VM (virtio)
VM to host (virtio)
OVS over VXLAN
L2 : OVS interconnect
V2.1 | 12 ©6WIND 2015
6WINDGate IP forwarding
performance
9.57 Mpps per core
Up to 226.83 Mpps with 22 cores
Performance scales linearly
with the number of cores
configured to run the fast
path.
Performance is independent of
frame size.
L3 processing
V2.1 | 13 ©6WIND 2015
L2
all VMs share the same broadcast L2 overlay
network
VM’s VLAN over OpenStack’s VXLAN
How to distribute MAC addresses, how to
manage L2 overlayed topologies?
Proxy ARPs
ISIS?
EVPN (BGP extension)?
PVSTP?
Live migration => MAC tables of switches to
refresh (GARP is not reliable)
L3
L3 interconnects: all IP traffic from VMs are
routed through its VRF such as VPNs
(RFC2547)
No broadcast domain
But L2 interconnect can still be supported:
Each tenant can configure its own L2 over IP
(VXLAN, NVGRE, MPLS over GRE, etc.)
VRF aware L3 routing is mature
Live migration => route updates
L2 or L3 interconnect ?
V2.1 | 14 ©6WIND 2015
Benefits of Neutron without L2
No OVS, no Linux Bridge
less networking bugs and less provisioning bugs
no VLAN
Less processing layers => Less CPU cycles
per packets => Increase packet rates
Simplify the design of PCI Ethernet boards
(HW offloads)
Why L2 is leading, but not L3 with OpenStack?
No L3 alternative to OVS
Quagga: EVPN not available
LISP: No proof points with Neutron
Like OVS, any solutions have to be
1st – democratized into Linux kernel, Linux networking
community
2nd – integrated use with OpenStack / Neutron
3rd – described for their usage through standard bodies (ETSI,
IETF, ONF, etc.)
OpenStack Neutron: L2 today, L3 tomorrow?
V2.1 | 15 ©6WIND 2015
Conclusion:
ETSI NFV does not assume L2 or L3 interconnect
SDN for VM2VM can be built based on L3 over L3 (aka VPNs)
L3 shall restore performance + simplicity