CloudStack Networking

Alex Huang

November 5 2012

Outline

• Design Goals

• CloudStack Terminology

• Architectural Model

• Networking APIs

Design Goals

• Enable networking partners to innovate and differentiate.

• Give control and choice to the cloud operator.

• Simplify presentation to the end user.

• Enable developers to concentrate on innovation.

Explosion of Network Features • L2

– Physical, VLAN, L3 (anti-spoof), Overlay[GRE], SDN. – QoS, traffic monitoring, broadcast & multicast.

• L3 – IPAM [DHCP], Public IP address management, Gateway, VPN,

Firewall, Static NAT, Source NAT, Site-to-Site VPN, L3 ACLs

• L4 – Security groups for L3-isolation, Stateful firewall for TCP, UDP

and ICMP, Port forwarding

• L7 – Loadbalancing, User-data, Password Change

• More will come – Key is CloudStack must not control innovation.

Enabling Innovation

• CloudStack must not define the innovation. – Partners define their own APIs. – Partners and CloudStack can work together on unified APIs

through design process on Apache.

• Differentiate between orchestration and provisioning. – CloudStack only orchestrates. – Provisioning is always pushed to the partner.

• Clearly defined data center abstraction layer. – Changes in this layer are broadcasted to partners.

• Utilize CloudStack’s orchestration to deploy and auto-scale partners’ technologies.

CloudStack Terminology (End User)

• Network – A single concept to encapsulate multiple network technologies to simplify

representation to the end user. – One Network to rule them all, One Network to define them, One Network to

bring them all and in the cloud bind them. – Each Network always carries its Network Traffic Type. – CloudStack DOESN’T understand how to provision this conceptual network on

to the physical network.

• NetworkService – L2-L7 network services that partners have written to operate within a

Network. – Currently defined: Load Balancing, Port Forwarding, Firewall, Gateway, DNS,

DHCP, Static NAT, VPN, Source NAT, User Data.

• NetworkOffering – A packaging of the NetworkServices provided to the end user on a particular

Network. – NetworkOfferings are put together by cloud operator.

CloudStack Terminology (Operator)

• Network Traffic Type – Traffic types are mapped to the underlying physical

network by the cloud operator. – Traffic type is not the same as network (Guest traffic type

can actually be carried on multiple networks) – Currently defined: Public, Guest, Storage (Backup really),

Management

• NetworkServiceProvider – Plugin that understands how to provide one or more

NetworkServices by using VPX or physical resource.

• PhysicalNetwork – Actual wiring of the data center.

CloudStack Terminology (Partner)

• NetworkGuru – Plugin that understands the network isolation technology, mac addressing scheme, and IP

addressing scheme deployed and how to map Network Traffic Types to the underlying physical network.

– CloudStack passes Network to NetworkGuru to “implement” before the network is needed by a virtual machine.

– CloudStack asks the NetworkGuru to issue ip, mac, and isolation to a virtual machine before it starts.

– CloudStack informs the NetworkGuru when a virtual machine stops so it can collect resources. – When all virtual machines in a Network are stopped, CloudStack garbage-collects the Network

by asking the NetworkGuru to shutdown the network. – CloudStack provides a default implementation for VLAN based isolation technology.

• NetworkElement – Interface that specifies the events CloudStack signals to the NetworkServiceProviders when a

Network needs to be “implemented” and shutdown and when a virtual machine joins and leaves a Network.

“Architect” Model

• The builder offers multiple blueprints for the owner to build the house. • Owner chooses on a blueprint and then adds on with additional enhancements

such as hardwood floors, granite counter tops, etc. • General contractor builds to the blueprint by orchestrating between different sub-

contractors to build different parts of the blueprint. • There are two general category of contractors.

– Rough-in sub-contractors who take care of plumbing, electricity, framing, foundation. – Finish sub-contractors who put in flooring, kitchen cabinets etc.

• Each sub-contractor is responsible for only their work but looks over the entire blueprint to make sure their work can actually be done. – E.g. A lighting plan may conflict or needs to change depending on the framing plan.

• General contractor is responsible for sequencing the sub-contractors to make sure everything the sub-contractor is dependent on is ready when the sub-contractor arrives to do his work.

• Every change requires a the blueprint to be republished so every sub-contractor can make their appropriate changes.

Comparison

Building a house

• Owner

• Builder

• General Contractor

• Rough-in Sub-Contractors

• Finish Sub-Contractors

• Blueprint

• Cabinets, Flooring, Counter Tops, etc

Building a network

• End user

• Cloud Operator

• CloudStack Orchestration

• NetworkGurus

• NetworkServiceProviders

• Network

• NetworkServices

Architectural Principles

• CloudStack clearly defines the difference between orchestration and provisioning. – Orchestration the ordering of what needs to happen in CloudStack’s

abstraction layer. – Provisioning is the actual work performed at the resource.

• CloudStack clearly defines the difference between network definition and network services. – Network definition is handled by NetworkGuru. – Network services is handled by NetworkServiceProvider.

• CloudStack broadcasts changes in the network every time NetworkServices and virtual machines changes in the Network.

• CloudStack allows the Cloud Operator to setup the appropriate mappings between virtual concepts such as Network and Network Traffic Type to the underlying physical network.

Kernel

Sequence Flow for VM Creation End User Rest API

Security Checkers

User VM Mgr

Network Mgr

Storage Mgr

Job Scheduling

VirtualMachine Mgr

Network Guru

Deploy VM

ACL Checks

Allocate Entity in CS

Allocate VM

Allocate NIC

Allocate Volume

Allocate IP

Schedules Deploy Job

Returns with job id, VM id

Query Job Result

Returns with job status

Sequence Flow for VM Creation Job Threads

Network Element

User VM Mgr

Network Mgr

Storage Mgr

VirtualMachine Mgr

Network Guru

Start VM

Start VM

Prepare Nics

Notify that Nic is about to be started in network

Reserve resources for Nic

Services API Server

Resources

Start User VM

Agent Calls

Prepare Volumes

Template Mgr

Deployment

Planner

Get a Deployment Plan (Host and StoragePool)

Prepare template on Primary Storage

Agent Calls

Agent Start VM Call

Stores job result

CloudStack User APIs [sample]

• Networks (L2)

– createNetwork [requires network offering id],

– deleteNetwork (A), listNetworks,

– restartNetwork (A): restarts all devices (if allowed) supporting the network and re-applies configuration

– updateNetwork: update network offering and restart network

• Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider

• Restarting the Network with “Clean up”: • restarting network elements - virtual

routers, DHCP servers

• If virtual router is used, it will be destroyed and recreated

• Reapplying all public IPs to the network provider

• Reapplying load-Balancing/Port-Forwarding/Firewall rules

Restarting and Cleaning Up a Guest Network

• An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged)

• Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool

Deleting a Guest Network

Extending CloudStack Networking

Network Manager

Network Element

DnsService

MyDnsElement MyDnsDeviceMa

nager

MyDnsDeviceService

PluggableService

MyDnsDeviceResource

AgentManager Queue

1. prepare (part of start vm) 2. prepare (Network, Nic, DeployDestination, VmInfo)

3. addDnsRecord(ip, fqdn)

4.Enqueue AddDnsRecord

5.API call to Dns Device

Device Configuration Admin API (CRUD)

MySQL

Needs to be added as of 5/2/2012

Demonstrates one way to inform an external DNS server when an instance starts. Classes shaded blue form a plugin / service bundle to integrate an external DNS server. Clients of the instance can then use DNS names to access the instance.