virtual networking for iaas cloudsonic.jp/archive/2012/material-sdn_japan_2012/7th... · virtual...

Virtual Networking for IaaS Clouds

Dan Mihai Dumitriu, [email protected] CTO, Midokura

SDN Japan - December 7, 2012

Copyright ©2012 Midokura All rights reserved

Requirements

2

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink

Provider Virtual Router (L3)

Tenant AVirtual Router

Tenant BVirtual Router

VM6

Virtual L2 Switch B1

Virtual L2 Switch A1


TenantB office

Tenant BVPN Router

Office Network


Requirements: recap

3

l  Multi-tenancy l  Scalable, fault-tolerant

devices (or device-agnostic network services).

l  L2 isolation l  L3 isolation

u  VPC, like VRF (virtual routing and fwd-ing)

l  L3 Gateway (BGP) l  L2 Gateway (STP, LACP) l  Scalable control plane

u  ARP, DHCP, ICMP l  Floating IP

l Stateful NAT

u  IP/Port masquerading u  DNAT

l ACLs l Stateful (L4) Firewalls

u  Security Groups l LB health checks l VPNs at L2 and L3

u  L2TP, IPSec l REST API l Integration with CMS

u  OpenStack u  CloudStack


MidoNet SDN Solution

4

Logical Topology

MidoNet Solution

15

Private IP Network

MN

MN

MN

Internet

BGP Multi

Homing

Physical Topology

MN VM

VM

MN VM

VM

MN VM

VM

BGP To ISP3

BGP To ISP2

BGP To ISP1

vPort

Provider Virtual Router

Tenant A Virtual Router

Tenant B Virtual Router

Virtual Switch A1

Virtual Switch A2

Virtual Switch B1

vPort

vPort

vPort

vPort

vPort

Network State Database

MN MN MN

Tunnel

Gateway Nodes Compute Nodes


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

5

Isolated tenant network (virtual

data center)


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

6

L3 isolation (similar to VPC and VRF)


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

7

Isolated L2 networks


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

8

Redundant, optimized and fault-tolerant paths to the

Internet (e.g. via BGP)


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

9

Fault-tolerant devices and links


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

10

NAT, LB, and Filtering

NAT, LB, and Firewalls


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

11

L3 (and L2) VPNs


Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

Requirements

12

Minimize ARP broadcasts by exploiting CMS config

RESTful API for CMS integration and direct

tenant access

Solid integration with leading open CMS:

OpenStack, CloudStack

DHCP, DNS and other services


How to build it?

13

1. Virtualized physical devices

2. Centrally controlled OpenFlow-based hop-by-hop switching fabric

3. Edge to edge overlays


Virtualized physical devices

14

l  4096 limit on number of unique tags l  Large spanning trees terminating on many hosts l  High churn in switch control planes due to MAC learning

l Each VM is separate virtual MAC! l  Need MLAG for L2 multi-path (vendor specific)

1

VLAN VLAN1

VLAN2



15

l  L2 isolation l  What about L3 and Internet access? l  Use VRF or virtual appliances? Fault-tolerance?

1

VLAN (more) VLAN1

VLAN2



16

1

l  Not scalable to cloud scale l  Expensive hardware l  Not fault tolerant (HSRP?) l  L2 and L3 isolation. What about NAT, LB, FW?

出典：http://infrastructureadventures.com/tag/vrf-lite/

VRF

Div.1 VLAN 10 VLAN11 VLAN12

Div.2 VLAN 20 VLAN21 VLAN22

Div.3 VLAN 99

VRF VRF VRF


OpenFlow hop-by-hop switch fabric

17

2

OpenFlow Switches

OpenFlow Controller (Cluster)

l  Fabric extends to the compute host software switch? •  State in each switch is proportional to the virtual network state •  Need to update all switches in path when provisioning new virtual

devices or updating them. •  Not scalable, slow and non-atomic switch updates.


OpenFlow hop-by-hop switch fabric (more)

18

2

OpenFlow Switches

OpenFlow Controller (Cluster)

l  Flow rules for VM flows (microflows)? l  Flow rules for virtual device simulation?


　Edge-to-Edge Overlays

19

3

VM

VM Edge

Edge Edge

Edge Edge

Edge

Use scalable IGP (iBGP, OSPF) to build multi-path underlay



20

3

VM

VM Edge

Edge Edge

Edge Edge

Edge

IP encapsulation provides isolation



21

3

VM

VM Edge

Edge Edge

Edge Edge

Edge

Virtual network processing at ingress host, decoupled from

physical network



22

3

VM

VM Edge

Edge Edge

Edge Edge

Edge

Virtual network changes don't affect

underlay state



23

3

• Packet processing on x86 CPUs (at edge) •  Intel DPDK facilitates packet processing •  Number of cores in servers increasing fast

• Clos Networks (for underlay) • Spine and Leaf architecture with IP • Economical and high E-W bandwidth

• Merchant silicon (cheap IP switches) •  Broadcom, Intel (Fulcrum Micro), Marvell •  ODMs (Quanta, Accton) starting to sell directly •  Switches are becoming just like Linux servers

• Optical intra-DC Networks


Spine and Leaf Network Architecture

24

L3 Switch Spine

Leaf

L3 Switch L3 Switch L3 Switch

L3 Switch L3 Switch L3 Switch x32

x4

48x10G

1536 x 10G

4x40G

e.g Force10 Z9000

IBGP and ECMP

e.g Arista 7050T

e.g Force10 Z9000


Overlays are the right approach!

But not sufficient... We still need a scalable control plane.

25



26

Distributed State

MidoNet REST API

Dashboard



27

Distributed State

Linux Kernel + OVS KMOD

VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Lazy state propagation



28

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

VM sends first packet; table miss; NetLink

upcall to MidoNet



29

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

MidoNet agent locally processes packet (virtual layer simulation); installs

local flow (drop/mod/fwd)



30

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Packet tunneled to peer host; decap; kflow table miss;

Netlink notifies peer MidoNet agent



31

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

MN agent maps tun-key to kernel

datapath port#; installs fwd flow rule



32

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Subsequent packets matched by flow rules

at both ingress and egress hosts



33

MN

Packet Encapsulated

Tunnel

Drop/Block

Packet from VM, VPN, or external BGP peer enters

kernel datapath



34

MN

Packet Encapsulated

Tunnel

Drop/Block One flow rule reflecting the outcome of the virtual layer

simulation AND the mapping of egress vport to peer host

decides to drop or fwd



35

• Distributed and scalable control plane Ø Handle all control packets at local MidoNet agent

adjacent to VM • Scalable and fault tolerant central database

Ø Stores virtual network configuration Ø Dynamic network state

² MAC learning, ARP cache, etc Ø Cached at edges on demand

• All packet modifications at ingress Ø One virtual hop

² No travel through middle boxes Ø Drop at ingress



36

• Scalable edge gateway interface to external networks • Multihomed BGP to ISP • REST API and GUI • Integration with popular open source cloud stacks • OpenStack • Removes SPOF of network node • Scalable and fault tolerant NAT for floating IP • Implements security groups efficiently

• CloudStack and Eucalyptus



37

Deep OpenStack Integration

• Quantum Plugin • L2 isolation, of course • Also… • L3 isolation (without VM / appliance) • Security groups (stateful firewall) • Floating IP (NAT) • Load balancing (L4)


HorizonWeb GUI

Quantum Plugin

MidoNet Manager (Web GUI)

Nova API

MidoNet API

RabbitMQ - Passing Queue

Compute Host

NovaCompute

Libvirt driver

Datapath

Inst

an

ce

A1

Inst

an

ce

B1

MidoNetAgent

Quantum

API

MidoNet Plugin

MidoNet Distributed

State

MidoNet Distributed

State

MidoNet Distributed

State

MidoNet EdgeAgent

MidoNet EdgeAgent

MidoNet EdgeAgent

INTERNET

MidoNetVIF Driver

KeystoneAuthentication

OpenStack Integration



39

Future Directions

• Scalable L7 virtual appliances • Content aware load balancer • MPLS VPN termination Ø Interconnect with carrier backbones • multiple data center federation Ø Virtual L2 between sites • LISP Ø Global IP mobility between sites



40

Conclusions

• IaaS clouds require new networking model • Edge to edge overlays are the right

approach • Servers are good enoug at packet

processing Ø Can use them for edge gateways • Multipath IP network fabric is cheap and

easy to build

Questions?

[email protected]

Midokura is hiring! in TYO, SFO, and BCN

[email protected]

virtual networking for iaas cloudsonic.jp/archive/2012/material-sdn_japan_2012/7th... · virtual...

Documents