www.huawei.com

Security Level:

HUAWEI TECHNOLOGIES CO., LTD.

WAN-TCTraffic Control for WAN links

Author/ Email: Ofer Ben-Yacov (ofer.benyacov@huawei.com)Version: V1.0(20161129)

Requirements Overview

Public Cloud Private CloudWAN Link

Project/ VM / Policy Based QoS on WAN

• In Hybrid Cloud scenario, there is a need to do rate limit on the WAN traffic• Different Projects can have different limits• Traffic from/to different source/destination within a Project can have different limit• Logical groups of VMs can be created (e.g. department) and be configured with different

limits• Hierarchical limitation support

• Limit Project to X• Limit VM/Group of VMs in that Project to Y with Y < X

• Future version. Will be done with P4 / XDP / eBPF

Hybrid Cloud Inter-Connectivity

Public Cloud Private Cloud

MPLS-VPN

L2GW

VPN

• Inter-Cloud connectivity can be with L2 or L3• Different devices/software can be used

• L2GW for L2• Software-based VPN (e.g. OpenSWAN) for L3• MPLS-VPN for L2/L3

Suggested Solution

Pass-Through Device No change is done to the network configuration Install the device in the path between private and public

clouds Usually before the WAN router

Limit will be done for VxLAN packets using their VNI Customer limit will be done using aggregation of VNIs

Public CloudPrivate

Cloud Internet

VNI 1

VNI 2VNI 3

RouterTC Device

Single Link Scenario

TC device will be install between the cloud devices (NNs, CNs) and the router that is the last device in the DC (the one that is physically connected to the WAN link)

Rate limit will be done to VxLAN packets which contains traffic to/from the public cloud

Public CloudPrivate

Cloud Internet

VNI 1

VNI 2

VNI 3

WAN-TC• Limit VNI 1 to 1Mbit• Limit VNI 2+3 to 5Mbit

• Limit VNI 3 to 2Mbit Router

Multi-Link Scenario Each one of the TC devices will be installed between the cloud

devices (NNs, CNs) and one of the WAN routers Limit will be configured the same way on the TC devices The scenario support active-active load balancing and fail over with

the restriction that each VNI traffic must use specific link (not split between links)

Public CloudPrivate

Cloud Internet

VNI 2

VNI 1

VNI 3

Router

Switch

WAN-TC• Limit VNI 1 to 1Mbit• Limit VNI 2+3 to 5Mbit

• Limit VNI 3 to 2Mbit

Load Balancing Scenario TC devices will be install between the cloud devices (NNs, CNs) and

the Load Balancer Limit will be done before the traffic will be split between the WAN

links

Public CloudPrivate

Cloud Internet

VNI 1

VNI 2Router

LB

WAN-TC• Limit VNI 1 to 1Mbit• Limit VNI 2 to 5Mbit

Software Components

Neutron Server

Neutron API Neutron API Extensions

WAN-TC Plugin

Agent Manager

TC DriverMessage Bus

WAN-TC API

• The plugin will be install on the NN• The Agent can be installed on any server

including on the TC device• Agent Manager can be installed on multiple

servers for fail-over• OVS or Linux bridge is used to pass traffic

from ingress to egress port• Limit is always done on egress port

OS

Data Path

OVS/Linux Br

Egress Qs

TC Device

TC Device

TC Device

Router

Router

Router

TC Device

Uses bridge such as OVS to connect LAN and WAN facing ports Use egress queues to limit traffic to/from external networks

OVS

Egress

Ingress

Port 1

Queues

Port 1

Egress

Ingress

Port 2

Queues

Port 2

Ingress Traffic

Egress Traffic

Linux QoS QoS in Linux in done using TC command

Traffic limiting in done in the egress port queues

To use the QoS we need to Set queue using tc qdisc Set class (limit) using tc class Choose the traffic to limit using tc filter

QoS exampleCreate the queue

tc qdisc add dev eth0 root handle 1: htb

Set limit to 1mbpstc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit

Choose traffic to limittc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip protocol 17 0xFF match u16 0x12b5 0xFFFF at 22 match u32 0x00006400 0xFFFFFF00 at 32 flowid 1:1

# this command select UDP+VxLAN traffic with VNI set to 100.# selected traffic will be sent to class 1:1 and be limited to 1mbps

TC API – Device ManagementThe following commands can be run for TC device List

wan-tc-device-list Show

wan-tc-device-show a16463a0-2549-45fb-bfdb-6c9d8b95ff61 Delete

wan-tc-device-delete a16463a0-2549-45fb-bfdb-6c9d8b95ff61 Create is done not but admin but by getting first message from the device

TC API – Class ManagementThe following commands can be run for TC class List

wan-tc-class-list Show

wan-tc-class-show 30e0ce8d-ab7c-40a9-9ba7-edce5f0c43d3 Delete

wan-tc-class-delete 30e0ce8d-ab7c-40a9-9ba7-edce5f0c43d3 Create

wan-tc-class-create <direction> --min <min> --max <max> [--parent parent]

TC API – Filter ManagementThe following commands can be run for TC filter List

wan-tc-filter-list Show

wan-tc-filter-show 30e0ce8d-ab7c-40a9-9ba7-edce5f0c43d3 Delete

wan-tc-filter-delete 30e0ce8d-ab7c-40a9-9ba7-edce5f0c43d3 Create

wan-tc-filter-create <protocol> <match> <class>

Security Level:

HUAWEI TECHNOLOGIES CO., LTD.

www.huawei.com

Thank You

Ofer Ben-Yacov