ODL Summit - DDF Seattle - Sep 2016NetVirt - Carbon Planning

Alon Kochba (alonko@hpe.com)Sam Hague (shague@redhat.com)

1

Agenda● TaaS (Tap-as-a-Service)● SNAT - decentralized, OVS nat based● IPSEC for VxLAN Tunnels, VPNaaS● Flow based VxLAN tunneling● Internal Tunnel Optimization Table - Review Pros/Cons● Proposed Pipeline Changes● QoS● Static and Dynamic Routes● LBaaS● Transparent VLANs● VLAN Aware VMs● FWaaS● DHCP - Controller based or OpenStack (qdhcp) based● IPv6● Containers● GRE Provider Networks● Etree and ELAN - refactoring● Multiple VxLAN endpoints for compute

2

● HWVTEP L3 Support● ToR switch configuration● SFC Integration● Traceroute Responder● Counters and logging for security

groups● Security groups on ToR ACLs● L2 BGP VPN● Clustering, HA● SR-IOV● Networking-odl v2● Dynamic ARP requests● CSIT, IT, UT

Features Delivered in Boron● Merge of NetVirt and VPNService projects● L2● L3● Auto-bridge creation● Auto-tunnel creation● Floating IP’s● VLAN and Flat provider network support for multiple internal

and external networks

3

● Security Groups○ Stateful using conntrack○ Stateless○ Learn (for OVS-DPDK)

● NAPT● SNAT● IPv6● Layer 2 Gateway

TaaS (Tap-as-a-Service) & Additional TAPNeutron TaaS

● Neutron extension○ https://github.com/openstack/tap-as-a-service

● Port mirroring of any neutron port to a “tap” port - this port can be any neutron port, for example a VM running Wireshark.

4

TAP● Additional TAP capabilities

○ Mirroring of provider network ports (VxLAN, VLAN etc - non neutron ports)

○ Mirroring of specific tunnel endpoints between OVS○ Mirroring of a tunnel traffic for a given network

● In all cases, TAP has to be highest priority service on a port (higher than ACLs), i.e the first service on VM egress, and last service on VM ingress

SNAT (NAPT)● Stateful Implementation

○ Use OpenFlow rules for SNAT instead of ODL packet-in.○ Implementation relies on OVS conntrack nat support.○ Will this be available in OVS 2.6?○ When will this be available for OVS-DPDK?

● Decentralized Implementation○ Use a unique IP address for each compute node, prevent traffic leaving

from compute node through a centralized “SNAT” switch.

● Cosmetics ○ Rename NAPT to SNAT - use OpenStack naming convention in code.○ Rename Floating IP SNAT and DNAT tables to something else to prevent

confusion.

● Details here - https://trello.com/c/edwnUPwx/50-l3-snat

5

IPSEC for VxLAN tunnels & VPNaaS● Look into supporting VPNaaS Neutron extension.● IPSec connectivity between computes for all network use cases (private

network, routing between networks etc.) - possibly not supported by VPNaaS, but can implement through a different Northbound (restconf).

● Authentication and key management?

● https://wiki.openstack.org/wiki/Neutron/VPNaaS● http://specs.openstack.org/openstack/neutron-specs/specs/api/virtual_pri

vate_network_as_a_service__vpnaas_.html

6

Flow based VxLAN tunneling● Genius Feature● Use a single VxLAN port endpoint - add OpenFlow rules for tunnel matching

and actions for setting tunnel destinations.● This should eliminate need for multiple VxLAN tunnel endpoints, greatly

simplifying OVSDB rules.● Conflicts with OVSDB BFD● What other monitoring can we do, BFD in ODL? Other ideas?

7

Internal Tunnel optimization (VxLAN)● Consider pros and cons of internal tunnel table:● Pros:

○ Optimized lookup, pipeline is fully processed on first node and has a shorter pipeline on target node.

● Cons:○ Data passed on line should not hold internal NetVirt datastore IDs.○ The VNI is currently used for passing metadata info - e.g. logical port ID,

ELAN IDs.○ The network segmentation ID should be used in the VNI per OpenStack.

This is more correct in terms of API and interoperability.○ This adds a new flow to the pipeline, logically complicating things.○ This is an asymmetric approach to other network types, e.g. VLAN and

Flat. This asymmetry also affects the possibility to generically handle all provider network types for purposes such as split horizon.

● Need to consider repurcussions on L3 pipeline carefully. MAC address re-writes need to be considered.

● Consider switching L2 initially, stabilize, followed by L3

8

Pipeline Changes● L2/L3 Services Pipeline Reversal

○ Use L2 before L3 services by default, instead of using a filter table in the beginning of L3 pipeline.

○ This is a more correct approach as discussed extensively, and avoids traditional networking issues.

○ Work to be done? Optional flag already implemented in Boron?○ Add example of SMAC learning of traffic destined to router interfaces.

● Output to ports should always happen through Egress table - currently output to tunnel ports breaks this assumption.

● Egress table should handle “split-horizon” related flows e.g. don’t output to port

● Missing MAC addresses changes in L3 tables - wrong for traditional networking scenarios

● Rename of table numbers - ease understanding and comprehension:○ Use a minimal amount of resubmits (only to dispatcher if possible)○ Use a logical grouping like the old netvirt, e.g.

■ 10-20 Ingress ACL■ 20-30 L2■ 40-50 L3

9

QoS● Is OVSDB based QoS fully supported in Boron?● Do we want to add functionality in Carbon?● OpenFlow Meter based QoS instead of OVSDB based?● http://docs.openstack.org/draft/networking-guide/config-qos.html

10

Static and Dynamic routing● Support openstack static route API

○ Neutron router-update --routes ...● Learning and publishing routing with external routers using OSPF

○ When configuring external networks, instead of a static configuration of the subnet and gateway, have an option to configure a router entity gateway and learn the subnets dynamically by speaking with it using OSPF.

● Other routing protocols - RIP? IS-IS?

11

LBaaS● Neutron extension● Load balancing between multiple neutron ports from a predefined pool, along

with health monitoring.● OpenStack has two implementations - Octavia based and agent based. Can

we simply support Octavia which uses a VM (Amphora)?● Rumors that legacy NetVirt had support for LBaaS - is this correct? If so, how

was it supported? Is there full Neutron-NB support?● Add OpenFlow rule based load balancing as an additional implementation for

ODL.● Support IP load balancing - currently the only supported protocols in LBaaS

are TCP, HTTP and HTTPS

● https://wiki.openstack.org/wiki/Neutron/LBaaS● http://docs.openstack.org/draft/networking-guide/config-lbaas.html

12

Transparent VLANs● Do we support this already?● https://trello.com/c/U53INVUQ/18-transparent-vlans

Transparent VLAN support is present for VxLAN based provider networksNeeds to be evaluated for VLAN provider networks

13

VLAN Aware VMs● https://trello.com/c/oG7b50BX/17-vlan-aware-vms ● API has been finalized in Openstack Newton● ELAN service has support for trunk and sub-ports● Need to add support in Neutron VPN bundle, co-ordinated with Neutron

project

14

FWaaS● Neutron extension● FWaaS operates at the perimeter by filtering traffic at the OpenStack

Networking (neutron) router. This distinguishes it from security groups, which operate at the instance level.

● Delayed to Ocata according to Carbon planning spreadsheet note?● https://wiki.openstack.org/wiki/Neutron/FWaaS/HowToInstall

15

DHCP● What approach do we want as our default DHCP?● Distributed DHCP?● OpenStack based DHCP (qdhcp):

○ Current default.○ Requires multiple qdhcp agents, tunnels to control nodes.

● Controller based DHCP:○ Can be enabled by controller-dhcp-enabled=true○ Can do smarter DHCP assignment, managed by a single server (ODL).○ Metadata - currently the metadata server(s) run on the control nodes,

and the qdhcp is the one who forwards packets to it. Removing it removes metadata support.

○ How can we support metadata (besides adding static route and keeping qdhcp)?

○ Any other pros/cons? Can we eliminate tunnels to control nodes and qdhcp agents if we use this?

○ Controller DHCP lacks IPv6 support○ Currently does not allocate its own Neutron port, should do this to not

confuse end nodes

16

IPv6● What’s still missing? Can we fully support IPv6 in Carbon?● IPv6 VxLAN tunnels?

17

Container support● OpenStack is planned to support running containers just like it runs VMs - see

OpenStack Zun.○ Make sure that networking functionality does not break.

● Container Orchestration Engine networking based on Neutron - Kuryr project○ Kuryr will support both Docker Swarm and Kubernetes○ We should make sure that ODL works correctly.○ Special requirements for Kuryr Kubernetes:

■ Support LBaaS for Kubernetes Service (http://kubernetes.io/docs/user-guide/services/)

● For efficiency support distributed LB that makes the LB decision at the source container/Pod.

● Service with sessionAffinity=true/false■ Might require that “compute nodes” will have access to running

containers.● Nested containers - OpenStack Magnum + Kuryr

○ For efficient container networking it is preferable that nested containers will get their networking directly from ODL.

○ Requires support for VLAN aware VMs.

18

GRE Provider Networks● For completeness of OpenStack network types.● Is this actually useful? Worth spending cycles on it?

19

Etree and ELAN - refactoring● Etree was implemented in the late stages of Boron.● We used a “dirty” design, to minimize risks for regressions in Elan.● Need to move to a cleaner design by refactoring Elan and Etree.● Suggestion made: Design the code as Etree is the general case, and Elan is an

Etree with only Roots.

Current design of Etree was “injected” into Elan like this:{

installFlowX();installFlowY();If (elanInstance.getAugmentation(Etree.class) == null) {

installElanBroadcastGroups();} else {

installEtreeBroadcastGroups();}

}

20

Multiple VxLAN endpoints on a compute● Support multiple tunnel end point IPs

○ Simple example IPv6 and IPv4 VxLAN endpoints○ Both end point connected to multiple networks○ Use BFD to support tunnel state

● Will be used in UniMgr - Not only through Neutron● How will this be modeled in Neutron?

○ Currently not supported

21

Need more details on proposed implementation

HWVTEP L3 Support● Daya to add details● This is required in order to allow not only L2GW on HWVTEP but also “L3GW”

which implies that the HWVTEP has the ability to forward based on IP addresses to the correct VXLAN tunnel (similar to DVR solution on L3 pipeline in OVS)

● The HWVTEP schema already supports such L3 table (see http://openvswitch.org/docs/vtep.5.pdf )

● Add support in the HWVTEP handler, to tie this into ELAN and L3 service logic● Support for ARP supression● Need to think of a solution for N-S traffic connectivity using different provider

types○ MPLSoGRE - not available on many devices○ VXLAN○ VLAN

23

ToR switch configuration● Allow NetVirt to manage ToRs connected to the OpenStack setup - e.g.

automatic provider network connections.● How is this modeled? What is the design/implementation details required?● Need a port map type information exchange between TOR and controller

24

SFC Integration● Sync with SFC project:

○ What is required to integrate SFC to use the new NetVirt?○ Are there any new features required from NetVirt?

● We do want the Neutron networking-sfc to work with the new NetVirt○ Needs to be independent from NSH in the VNF ○ See http://docs.openstack.org/developer/networking-sfc/api.html

● Implement Neutron SFC APIs (networking-sfc)

25

Traceroute Responder for Router Interfaces● Similar to pings, but with traceroute capabilities, needed for figuring out

connectivity breaks in an external network or vpn connectivity scenario for VMs

● Can be applicable to debug NAT issues, security group policy, vpn connectivity issues etc.

26

Counters and logging for security groups ● Ability for operators to track and monitor security activity, with feedback from

the system, that can be hooked into external alarms and traps● Mechanisms to track the effectiveness of security group rules, with the right

set of counters● Apis to define thresholds to trigger traps● Do we make these configurable ? how should they be stored, etc.

27

Security groups service instantiation on ToR ACLs

● security groups should work seamlessly across workloads connected to TOR VTEPs

● May be implemented using the ACL tables of HWVTEP devices for intra-DC● Multitenancy support not available in the tables● Stateful implementation not possible● N-S ACLs on NATted pkts not possible● Alternatives ? - designated vswitch hops (similar t NAT?)

28

Ethernet VPN support (i.e., L2 BGPVPN)● Support for evpn based connectivity to TOR VTEPs and DC-GW entitities● Related RFCs-● · https://tools.ietf.org/html/rfc7432

● · https://www.ietf.org/id/draft-ietf-bess-evpn-overlay-04.txt

● · https://tools.ietf.org/html/draft-boutros-bess-vxlan-evpn-01

● · https://tools.ietf.org/html/draft-ietf-bess-evpn-prefix-advertisement-02

● Controller will exchange L2 + VTEP information (RT2) with TOR VTEPs as well as DC-GW to stretch the L2 network across locations

● L3 prefixes can also be exchanged using RT5 to implement inter-subnet forwarding using EVPNs

● Dataplane will use VXLAN instead of MPLSoGRE for inter-DC

29