dragonflow 01 2016 tlv meetup
TRANSCRIPT
What is Dragonflow?
Full Implementation of OpenStack Neutron API
Lightweight Distributed SDN Controller with pluggable database
Project missionTo Implement advanced networking services in a manner that is efficient, elegant and resource-nimble
Page 2
Dragonflow Highlights
Page 3
• Integral part of OpenStack
• Fully Open Source
• Scale, Performance and Latency
• Lightweight and Simple
• Easily Extendable• Distributed SDN Control Plane
• Sync Policy Level abstraction to the CN
Dragonflow - Distributed SDN
Neutron-Server
Dragonflow Plugin
DB
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
OVSDragonflowDB
Driver
Compute Node
OVSDragonflowDB
Driver
Compute Node
DB
VM VM..VM VM..
VM VM.. VM VM..
Compute Node Compute Node Compute Node Dragonflow
Network DB
OVS
NeutronServer
OVSDB
OVSDB-Server
ETCD RethinkDBRAMCloud
Kernel Datapath Module
NIC
User Space
Kernel Space
Dragonflow DB DriversOVSDB ETCD RethinkDBRMC
Future
Dragonflow PluginRoute Core
API SG
vswitchd
Container
VM Dragonflow ControllerAbstraction Layer
L2 App L3 AppDHCP App
FaultDetection
SG
LBaaS …FWaaS
Pluggable DB Layer
NB D
B Dr
iver
s
SB DB Drivers
smartNIC OVSDB
OVSDB
ETCD
RMC
RethinkDB
OpenFlow
Dragonflow – Under The Hood
Current Release Features (Liberty)L2 core API, IPv4, IPv6
GRE/VxLAN/Geneve tunneling protocols
Distributed L3 Virtual RouterHybrid proactive + reactive flow installationNorth-South traffic is still centralized
Distributed DHCP (with just 500 lines of code!)
Pluggable Distributed DatabaseETCD, RethinkDB, RAMCloud, OVSDB
Network Node
DHCP namespace
DHCP namespace
DHCP namespace
DHCP namespace
Neutron DHCP Implementation
DHCP namespace
dnsmasq
DHCPAgent
Neutron Server
Message QueueExample• 100 Tenants• 3 vNet / tenant= 300 DHCP Servers
1 VM Send DHCP_DISCOVER
2 Classify Flow as DHCP, Forward to Controller
3 DHCP App sends DHCP_OFFER back to VM
4 VM Send DHCP_REQUEST
5 Classify Flow as DHCP, Forward to Controller
6 DHCP App populates DHCP_OPTIONS from DB/CFG and send DHCP_ACK
Dragonflow Distributed DHCP
DHCP DISCOVER
VM DHCP SERVER
DHCP OFFER DHCPREQUEST
DHCPACK
13
46
7
Compute Node
Dragonflow
VM
OVS
VM
1 2
br-intqvoXXX qvoXXX
OpenFlow
14
25
7
Dragonflow ControllerAbstraction Layer
L2App
L3App
DHCPApp SG
36
Pluggable DB Layer
DB
Dragonflow Distributed DHCP
Match: Broadcast +UDP +S_Port=68 +D_Port=67
Action:Send to DHCP table
Service Table
DHCP TableMatch: in_port => Action:
Set metadata with port unique keySEND TO CONTROLLER
(for every local port that its network has DHCP enabled)
Default:goto “L2 Lookup Table”
Compute Node
VM
OVS
br-intqvoXXX
VM
qvoXXX
1 2
DragonflowDragonflow Local Controller
Abstraction Layer L2
AppL3
AppDHCP App SG
DB
OpenFlow
Ingress Port SecurityIngress ClassificationDispatch to Ports
Database FrameworkRequirements• HA + Scalability• Different Environments have different requirements
• Performance, Latency, Scalability, etc.
Why Pluggable?• Long time to productize• Mature Open Source alternatives• Allow us to focus on the networking services only
DB Driver APIImplementations
RAMCloud
ETCD
RethinkDB
Zookeeper
Dragonflow Pluggable Database
Compute Node Compute Node Compute Node
DragonflowLocal
Controller
PluggableDB Layer
Applicative DB LayerAdapter
DBDriver
API
Expose DB Features
Neutron ServerDragonflow
Neutron Plugin
DB Operations
DatabaseServer
DB Adapter
DB Adapter
DB Adapter
DistributedDatabase
DB Data 3DB Data 2DB Data 1
Full Distribution
Compute Node 1
DragonflowLocal Cache
OVS
Compute Node NDragonflow
OVS
Local Cache
Dragonflow DB DriversOVSDB ETCD RethinkDBRMC
DB Data 3DB Data 2DB Data 1
DB Data 3DB Data 2DB Data 1
DistributedDatabase
DB Data 3DB Data 2
DB Data 1
Selective Proactive Distribution
Compute Node 1
DragonflowLocal Cache
OVS
DB Data 1
Compute Node NDragonflow
OVS
Local Cache
DB Data 3DB Data 2
Dragonflow DB DriversOVSDB ETCD RethinkDBRMC
Selective Proactive Distribution
Compute Node 1
DragonflowLocal Cache
OVS
Net1 – VM1, VM2
Compute Node 2Dragonflow
OVS
Local CacheNet2 – VM3, VM4
VM1 VM2 VM3 VM4
RethinkDB
Net2 – VM3, VM4Net1 – VM1, VM2
Dragonflow PipelineInstalled in every OVS
Service TrafficClassification
Ingress Processing
(NAT, BUM)
ARP DHCP
L2Lookup
L3LookupDVR
EgressDispatching outgoing traffic to external nodes or local ports
Ingress Port Security(ARP spoofing , SG, …)
EgressPortSecurity
EgressProcessing
(NAT)
Fully Proactive
Has Reactive Flows to Controller
Security Groups
…
Outgoing from local port Classification and tagging
Dispatching Incoming traffic from external nodes to local ports
Roadmap Additional DBs Drivers ZooKeeper, Redis …
Selective Proactive DB Hierarchical Port Binding (SDN ToR) move to ML2 Pluggable Pub/Sub Mechanism DB Consistency Distributed DNAT Security Group Containers (Kuryr plugin and nested VM support) Topology Service Injection / Service Chaining Inter Cloud Connectivity (Border Gateway / L2GW) …
Hierarchical Port Binding (SDN ToR) move to ML2
Rack n
ToR
VLANSegmentation
Rack 1
ToR
Rack 2
ToR
Rack 3
ToR
VxlanSegmentation
Dargonflow Hierarchical Port Binding (SDN ToR)
Neutron Server
REST API
Neutron Core plugins
ML2
Cisc
o (N
exus
, N
1Kv)
OVN
Mor
e ve
ndor
pl
ugin
s
Type Drivers Mechanism Drivers
VLAN
GRE
VXLA
N
ON
OS
Drag
onflo
w
TOR
Neutron Service plugins
Drag
onflo
wDB
Rack n
ToR
VLANSegmentation
VxlanSegmentation
Compute Node
Dragonflow
VM
OVS
VM
br-intqvoXXX qvoXXX
OpenFlow
Dragonflow ControllerAbstraction Layer
Vlan L2
AppL3App
DHCPApp SG
Pluggable DB Layer
DBDB
ToR
Mac
h Dr
iver
Ope
nDay
Ligh
t
Pluggable Pub/Sub Mechanism Neutron-Server
Dragonflow Plugin
DB
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
OVSDragonflowDB
Driver
Compute Node
OVSDragonflowDB
Driver
Compute Node
DB
VM VM..VM VM..
VM VM.. VM VM..
Pub/Sub
if the DB internally supports Pub sub then we use it
Pluggable Pub/Sub Mechanism
Neutron-Server
Dragonflow Plugin
DB
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
DB
VM VM..
VM VM..
VM VM.. VM VM
..
Pub/Sub
Why do we need it ? Not all DBs support pub-sub (e.g. RamCloud) We need to be able to customize
Performance, Latency, Scalability, etc.
DB Consistency Common Problem to all SDN Solution
SDN ControllerNorth-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
NeutronDB
Neutron-serverML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-PluginService
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova ComputeVM VM
Nova ComputeVM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron Plugin Agent
Neutron Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
Load
Bal
ance
r
Fire
wall
VPN
L3 S
ervic
es
Topo
logy
Mgr
.
Ove
rlay
Mgr
.
Secu
rity
Dragonflow DB ConsistencyNeutron-Server
Dragonflow Plugin
DB
OVSDragonflow
DBDriver
Compute Node
OVSDragonflow
DBDriver
Compute Node
DB
VM VM.. VM VM
..
NeutronDB
The Neutron DB is the master Database Introduce a full-sync diff based mechanism NDB
DDB Introduce a virtual transaction mechanism NDB
DDB
Key DB Requirement from multi production environments Optimized for Read, multiple read request in
very high volume from nova, Horizon … Multi Neutron server API running on different
hosts
Neutron-Server
Dragonflow Plugin
DB
Neutron-Server
Dragonflow Plugin
DB
Join the project Dragonflow
• Documentation https://wiki.openstack.org/wiki/Dragonflow• Bugs & blueprints https://launchpad.net/dragonflow• DF IRC channel #openstack-dragonflow
Weekly on Monday at 0900 UTC in #openstack-meeting-4 (IRC)
Security Groups Problems• Data plane performance
• Additional Linux Bridge on the Path
• Control plane performance• Rules needs to be re-compiled on port changes
• Many rules due to security group capabilities• Iptable commands issued by CLI process• RPC bulks
Security Groups Translations
Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote IP Prefix:0.0.0.0/0
match:ct_state=+new+trk,tcp,reg6=X actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)
Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote Security Group: Y
match:ct_state=+new+trk,tcp,reg6=X,reg5=Y, actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)
Distributed DNAT (Floating IP)
OVS
VM
Compute Node
Public network
OVS
VM
Compute Node
Public network
OVS
Network Node
RouterNamespace
Neutron and libnetwork
A Docker Container
Network Sandbox
Endpoint
A Docker Container
Network Sandbox
Endpoint
A Docker Container
Network Sandbox
Endpoint
Frontend Network
Endpoint
Backend Network
Tenant A Net1192.168.1.0/0
Tenant A Net2192.168.5.0/0
VM1192.168.1.5
VM2192.168.1.7192.168.5.2
Kuryr Project Overview• Open source • Part of OpenStack Neutron’s big stadium
• Under OpenStack big tent from next release!!!
• Brings the Neutron networking model as a provider for the Docker CNM• Aims to support different Container Orchestration Engines
• E.g. Kubernetes, Mesos, Docker Swarm
• Weekly IRC meetings• Working together with OpenStack community
• Neutron, Magnum, Kolla
Dragonflow and Kuryr plans• Dragonflow to support containers networking use cases
• Nested containers inside VMs support• Containers can leverage all of Dragonflow features
• Distributed DHCP• Security and QoS
• Containers performance and fault management• Port forwarding• Dragonflow distributed load balancer• DNS as a Service in Dragonflow• Integration with Kubernetes
• Full Integration of Dragonflow and Kuryr• Containerized image of Dragonflow• VIF Binding to Dragonflow
• OVS, IPVLAN
Mixed OpenStack Environments
Neutron network 1Neutron network 2Neutron network 3
Compute Node
VM
Dragonflow OVS(Controller: Dragonflow)
IPVLAN / OVS
VM
Dragonflow Service Injection
Dragonflow PipelineTenant/Admin Added Services
DynamicRouting VPN Firewall
DPI QoS LB
ServiceDiscovery
InterCloud
Public Network10.50.50.0/24
Router Router
Tenant A Net1192.168.1.0/0
Tenant A Net2192.168.5.0/0
Tenant B Net1192.168.1.0/0
Tenant B Net2192.168.9.0/0
VM1192.168.1.5
VM2192.168.1.7192.168.5.2
VM3192.168.1.9
VM1192.168.1.3
VM2192.168.1.3192.168.9.5
VM3192.168.9.7
Neutron Abstractions
Simple But Extendable• Various special services and behavior's
• VPN• QoS (DSCP tagging)• Dynamic Routing• Inter clouds connectivity• And so much more…
• External applications • Centralized “SDN” applications• New distributed networking services
• Networking as a Service to NFV
Classic Service Chaining• Chain of ports the traffic traverses
• Classifier for entry point
• Different types of chains• Static or dynamic
• Different underlying technologies• NSH• MPLS• App ports
• End points of various kinds• VMs• Containers• User space applications• Physical devices
Topology-based Service InjectionCompute Node
VM 1 VM 2
Table 0 Table 1 Table N…
External Application
External Application
Table
OpenFlow / Other API
OVS
Service Injection HooksLogical Router
Logical Switch Logical Switch
VM 1 VM 2 VM 3
DSCP Marking
DPI
DistributedLoad
Balancing
Topology Service Injection• Interact with base OpenFlow pipeline
• Leverage classification metadata
• Distributed network services • Flow based
• Compatible with SDN Applications• Can use OpenFlow
• Expose virtual topology• Inject services in specific hooks
• Easily extendable• No code modifications
Service Injection Example – IPS
Compute NodeVM 1 IPS
Table 0 ServiceChains Table N…
IPS Manager
Data Path App IPS recognizes infected VM
Service Injection Example – IPS
Compute NodeVM 1 IPS
Table 0 ServiceChains Table N…
IPS Manager
Data Path AppIPS app manager installs blocking flows for VM1 traffic (Quarantine)
Use Cases• Security Appliance
• Send specific traffic for inspection
• Traffic Mirroring • Implement TAP on various different locations in the path
• Applicative Load Balancing• Receive first packets of a connection and wire connection in flows
• Tenants Differentiate service between clouds• Inter Cloud connectivity
• Border Gateway / L2GW
Server Server
Detect Elephant Flows
0 1 … 64Flow Table
Test 110.0.0.3
Test 210.0.0.4
0 1 … 64Flow Table
Elephant detector
Detect elephant flow:10.0.0.3 10.0.0.4 TCP port 1234
Write flows to tableDSCP=64
slow path
fast path
CollectsFlowstats
Dragonflow Inter Cloud Connectivity (Border Gateway)
CN
CN
CN
NN
CN
CN
CN
NN
Data Center B
GW-GWTunnel
Data Center AIntra-CloudTunnels
Intra-CloudTunnels
ConnectingBare-Metal Servers as before
192.168.10.2
192.168.10.3192.168.10.8