SDN primerOvS

NVP

Networking in virtual environments

Guillaume Urvoy-Keller

January 7, 2018

1 / 36

SDN primerOvS

NVP

Source documents

Teemu Koponen, Keith Amidon, Peter Balland, Martín Casado,Anupam Chanda, Bryan Fulton, Igor Ganichev, Jesse Gross, PaulIngram, Ethan J. Jackson, Andrew Lambeth, Romain Lenglet,Shih-Hao Li, Amar Padmanabhan, Justin Pettit, Ben Pfaff, RajivRamanathan, Scott Shenker, Alan Shieh, Jeremy Stribling,Pankaj Thakkar, Dan Wendlandt, Alexander Yip, Ronghua Zhang:Network Virtualization in Multi-tenant Datacenters. NSDI 2014:203-216

Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan J. Jackson, AndyZhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer,Pravin Shelar, Keith Amidon, Martín Casado: The Design andImplementation of Open vSwitch. NSDI 2015: 117-130

2 / 36

SDN primerOvS

NVP

Outline

1 SDN primer

2 OvS

3 NVP

3 / 36

SDN primerOvS

NVP

Traditional networks

Strict layeringLayer 2 : VLANsLayer 3 : routing between VLANsMiddleboxes (NAT, Firewalls, IDS ) operate at layer 4 and above,e.g., check TCP port or application info (e.g., HTTP header)

Relies on distributed algorithms (spanning tree, routing protocols)

You don’t control their convergence, e.g., spanning tree prunessome links to avoid loops and elects a master or you assignsweights to OSPF but can’t impose a rootMPLS allows virtualization if links and actual path control

4 / 36

SDN primerOvS

NVP

Software Defined Networking

"One ring to rule them all" ⇒ centralized control plane, a.k.a,controllerController injects rules in switches and can read stats

If a switch does not a rule for a flow, it asks the controller

Rules are more complex and can mix layer 2 to 4 attributes (e.g.,if src MAC is xxx and TCP port is yyy, then) + meta-data info likeinput port.

OpenFlow v1.0 header fields Ingress Port, Ethec Src, Ether Dst,Ether Type, Vlan ID, IP Dst, IP Src, TCP Dst, TCP Src, IP Proto.

A rule is filter and an action :forward, discard, send to controlleror modify packet (e.g., modify @ like NAT)

5 / 36

SDN primerOvS

NVP

Software Defined Networking

6 / 36

SDN primerOvS

NVP

Software Defined Networking

SDN enables fine grained traffic control of traffic

Protocol to inject between controller and switches is normalized⇒ Openflow

Major vendors (HP, CISCO, etc) have released hardware switches

Also virtual switches like Open vSwitch (OvS)

Variety of Openflow controller: Floodlight, Opendaylight, ...

7 / 36

SDN primerOvS

NVP

Open vSwitch

8 / 36

SDN primerOvS

NVP

Open vSwitch

Borrowed slides from Ben Pfaff. See online talk athttps://www.usenix.org/conference/nsdi15/technical-sessions/

presentation/pfaff

What is OvS (from openswitch.org) ?

“Open vSwitch is a production quality, multilayer virtual switch licensedunder the open source Apache 2.0 license. It is designed to enablemassive network automation through programmatic extension, whilestill supporting standard management interfaces and protocols (e.g.NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag).”

9 / 36

SDN primerOvS

NVP

Where is OvS used?

Broad support:Linux, FreeBSD, NetBSD, Windows, ESX KVM, Xen, Docker,VirtualBox, Hyper-V, ... OpenStack, CloudStack, OpenNebula, ...

Widely used:

Most popular OpenStack networking backendDefault network stack in XenServer1,440 hits in Google ScholarThousands of subscribers to OVS mailing lists

10 / 36

SDN primerOvS

NVP

OvS architecture

11 / 36

SDN primerOvS

NVP

OvS architecture

ovs-vswitchd:userland daemontalks Openflow with controllerEssentially the same for all OS

datapath kernel moduleOS dependent + technology dependent, e.g., DPDKFrom http://www.dpdk.org/:These libraries can be used to:receive and send packets within the minimum number of CPU cycles (usually less than 80 cycles)develop fast packet capture algorithms (tcpdump-like)

run third-party fast path stacks

ovsdb-server:Stores configuration of switchesOpenflow does not allow to create/delete switches. ovsdb doesthis job!

12 / 36

SDN primerOvS

NVP

Packet data path

Kernel Datapath

"the datapath module simply follows the instructions, called actions,given by ovs-vswitchd, which list physical ports or tunnels on which totransmit the packet"

Datapath does not talk/is not aware of Openflow (this is theovs-vswitchd job)

13 / 36

SDN primerOvS

NVP

Packet data path

The more you do in the kernel, the better (faster) it is.

14 / 36

SDN primerOvS

NVP

15 / 36

SDN primerOvS

NVP

Network virtualization (NVP - see next section on VMware)

Mutli-tenant architecture

Each tenant expresses its network architecture in the form of aset of tables to traverse

Each table corresponds to a function (NAT, layer 2, routing, etc)

Each line in the table is an openflow rule

16 / 36

SDN primerOvS

NVP

Implementation of table

Hardware SDN switches benefit from TCAM memory

TCAM = Ternary Content Addressable Memory

CAM is an hardware implementation of an associative array

CAMCAM is a memory that can do memory lookups in one clock cycle andin a parallel fashion looking at multiple fields at once in a lookup.

17 / 36

SDN primerOvS

NVP

Binary and Ternary CAMs

BCAM outputs a 0 or a 1

Figure: source:https://www.sdxcentral.com/articles/contributed/sdn-openflow-tcam-need-to-know/2012/07/

18 / 36

SDN primerOvS

NVP

Binary and Ternary CAMs

TCAM further supports 0, 1 or ’don’t care bit’.Allows to account for variable size inputs to be hashed, e.g., IPprefixes of different sizes 192.168.1.0/24 and 192.168.1.0/25

Figure: source:https://www.sdxcentral.com/articles/contributed/sdn-openflow-tcam-need-to-know/2012/07/

19 / 36

SDN primerOvS

NVP

How do we do in software?

Problem: x86 architecture does not feature TCAM but simpleRAM.

We have efficient hashing functions but the keys must have thesame length

For OvS, they use Tuple search classifiersV.Srinivasan, S.Suri, and G.Varghese. PacketClassification Using TupleSpace Search. In Proc. of SIGCOMM, 1999.

20 / 36

SDN primerOvS

NVP

Tuple packet classification

A tuple is a known set of bits in each input fieldsEx: assume rules only use IP source and destination and thereare 2 different prefix lengths /8 and /24 ⇒ this gives 4 tuples:

IP source with 8 bits + IP dest with 8 bitsIP source with 24 bits + IP dest with 8 bitsIP source with 8 bits + IP dest with 24 bitsIP source with 24 bits + IP dest with 24 bits

A tuple search can be implemented as a hash function

We benefit from the fact that each field (IP @, MAC @, layer 4ports) feature in practice a limited number of different lengthsEx: you don’t have all /x addresses for x ∈ {1,2, ...32} but maybeonly {8,16,32}

21 / 36

SDN primerOvS

NVP

22 / 36

SDN primerOvS

NVP

Improving tuple search performance

100 lookup (not unusual in practice in an NVP implementation)too long at high rate, several 100s of Mb/s to Gb/s

Solution: pay the price for the first packet and cache result inkernel datapath for subsequent packet of the same layer 4connection ⇒ a single hash for packet number 2,3,...

A layer 4 connection is called a micro-flow in OvS parlance

23 / 36

SDN primerOvS

NVP

Microflow caching

Going to controller to ask the rule is not consider has an option in realimplementation ⇒ pro-active (install rules in advance) rather thanreactive model! 24 / 36

SDN primerOvS

NVP

Microflow caching

In practice, performance has improved but they use other techniques(called mega-flow) in practice

25 / 36

SDN primerOvS

NVP

Network Virtualization Platform (NVP)

26 / 36

SDN primerOvS

NVP

VMware & Nicira, Nicira Network Virtualization Platform(NVP)

Slides, article and presentation at:https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/koponen

Nicira: a startup that developed network virtualization tools

Bought by VMware in 2012 ⇒ NVP and now NSX (The NetworkVirtualization and Security Platform) - seehttps://www.youtube.com/watch?v=a1Ug9VomSvM&index=12&list=PL0DJsUAD5fbrzhfcnR-ZT7L3tqDX7q3mr

27 / 36

SDN primerOvS

NVP

Network virtualization already exists

28 / 36

SDN primerOvS

NVP

What if we use those legacy tools....

What if two tenants want to use the same set of private addresses, say10/8?1

⇒ need to decouple IP @ space of clients from the one of the physicalones.

1VRF might help but you have to be cautious in dynamic environments wheretenants provision their VMs/networks by themselves 29 / 36

SDN primerOvS

NVP

Decoupling physical from logical network

Similarly to what (OS) hypervisors do.Enable tenants to reproduce their network with architecture andsecurity constraints

30 / 36

SDN primerOvS

NVP

Your constraints when building a network hypervisors forclients to share the network

VMs must not be aware that there is not a physical but a logicalnetwork ⇒ same TCP/IP stackClients must be able to express their architectural/securityconstraints

There is no one single control plane? (CISCO CLI, JunOS, firewallspecific interface....)VMware vision: tenants’ needs can always be expressed asdatapath, a set of tables containing rules

31 / 36

SDN primerOvS

NVP

Generality of datapath model

32 / 36

SDN primerOvS

NVP

Where to implement?

Inside the virtual switches hosted in each hypervisor ⇒ nohardware support.

Tenants use an API to instruct the network hypervisor, e.g.,Openstack GUI to specify network architecture and a driver isused with the network hypervisor

33 / 36

SDN primerOvS

NVP

Inside the virtual switch

Significant burden for virtual switches that implement the wholedatapath of the tenant

34 / 36

SDN primerOvS

NVP

Physical network (between hypervisors)

Physical layer is kept simple and stupid ⇒ mesh of IP tunnels betweenphysical IP addresses of hypervisors

They bump into known performance models for tunnels like GREProblem stems from difficulty to perform TCP checksumoffloading to NIC.Use of STT

35 / 36

SDN primerOvS

NVP

Challenge of NVP controller

⇒ Cluster of controller to compute and maintain states (and othertechniques)

36 / 36