apache cloudstack networking by chiradeep vittal
DESCRIPTION
Apache CloudStack is a mature IAAS platform designed for scale and ease-of-use. However new cloud administrators typically struggle with networking in Apache CloudStack. Networking in CloudStack is full-featured, full of bells and whistles and by necessity complicated. This session will take the audience through the ins-and-outs of CloudStack Networking. Attendees will learn the motivations behind how CloudStack networking is architected, solutions to common networking requirements and future work. About Chiradeep Chiradeep Vittal is Distinguished Engineer in the Networking and Cloud Group at Citrix Systems. He is a maintainer in the Apache CloudStack project where he contributes to networking and storage parts of the Infrastructure-as-a-Service (IAAS) management system. He was a founding engineer at Cloud.com whose product CloudStack is now Apache CloudStack. CloudStack is deployed in more than 300 public and private clouds and powers some of the largest clouds in the world today.TRANSCRIPT
Introduction to CloudStack Networking
Silicon Valley CloudStack Meetup9th October 2014
About me
Agenda• Introduction to CloudStack• Networking modes in CloudStack• Virtual Networking• Networking Internals• Advanced Topics
Apache CloudStack is a• scalable, • multi-tenant, • open source, • purpose-built,• cloud orchestration platform for • delivering turnkey Infrastructure-as-a-Service
clouds
Apache CloudStack
300+ Large Scale
Production CloudsIn Deployment
Production sites with over
40,000+Servers Web
2.0
Service Providers and Telcos
Enterprise and Education
How did Amazon build its cloud?
Commodity Servers
Commodity Storage
Networking
Open Source Xen Hypervisor
Amazon Orchestration Software
AWS API (EC2, S3, …)
Amazon eCommerce Platform
How can YOU build a CloudStack cloud?
Servers StorageNetworking
Open Source Xen Hypervisor
Amazon Orchestration Software
AWS API (EC2, S3, …)
Amazon eCommerce Platform
Hypervisor (XenServer/KVM/vSphere/Hyper-V/L
XC)
CloudStack Orchestration Software
Optional Portal
CloudStack or AWS API
Secondary StorageImage
L3/L2 core
DC Edge
End users
Pod Pod Pod Pod
Zone Architecture
Pod
Access Sw
MySQL
CloudStack
Admin/User API
Primary StorageNFS/ISCSI/FC
Hypervisor (Xen/VMWare/KVM)
VM
VM
Image
Disk Disk
VM
VM
End users
Pod Pod Pod Pod
Networking concerns in a cloud
Pod
VM
VM
Disk Disk
VM
Networking Concerns• Network virtualization
– Multi-tenancy• Network services for virtual networks and
machines• Network automation• Scalability
Networking Principles in Apache CloudStack
• Flexibility– Allow various combinations of technology for L2-L7 network services– Allow different providers (vendors) for the same network service in a
Cloud POP• Pluggability
– Plugins allow vendors to drop in vendor-specific configuration and lifecycle management code
• Service scalability– Scale out using virtual appliances when possible– Scale up using hardware appliances if needed
Network FlexibilityNetwork Services
• L2 connectivity• IPAM• DNS• Routing• ACL• Firewall• NAT• VPN• LB• IDS• IPS
Network Isolation
• No isolation• VLAN
isolation• Overlays• L3 isolation
Service Providers
Virtual appliances
Hardware firewalls
LB appliances SDN
controllers IDS /IPS
appliances VRF Hypervisor
Networking Modes• “Basic” mode
– L3 isolation– Tenants share subnets– VMs placed into security groups
• ACL governs communication between/within groups/outside
– No VLANs– Excellent scaling (10s of thousands of hosts/VM)– Limited network services– Distributed network firewall using iptables on the hypervisor
…
DB Security Group
WebSecurity Group
Layer 3 cloud networking
… …
Web VM
Web VM
Web VM
Web VM
DB VM
Web VM
DB VM
Web VM
Ingress Rule: Allow VMs in Web Security Group access to VMs in DB Security Group on Port 3306
L3 isolation with distributed firewallsTenant 1 VM 1
10.1.0.2
Tenant 2 VM 1
10.1.0.3
Tenant 1 VM 2
10.1.0.4
Public Internet
10.1.0.1
Public IP address 65.37.141.1165.37.141.2465.37.141.3665.37.141.80
Load Balancer
L3 Core
Pod 1 L2 Switch
Pod 3 L2 Switch
10.1.16.1
…
…10.1.8.1Pod 2 L2 Switch
L3 isolation with distributed firewallsTenant 1 VM 1
10.1.0.2
Tenant 2 VM 1
10.1.0.3
Tenant 1 VM 2
10.1.0.4
Tenant 1 VM 3
10.1.16.47
Tenant 1 VM 4
10.1.16.85
Public Internet
10.1.0.1
Public IP address 65.37.141.1165.37.141.2465.37.141.3665.37.141.80
Load Balancer
L3 Core
Pod 1 L2 Switch
Pod 3 L2 Switch
10.1.16.1
…
…10.1.8.1Pod 2 L2 Switch
L3 isolation with distributed firewallsTenant 1 VM 1
10.1.0.2
Tenant 2 VM 1
10.1.0.3
Tenant 1 VM 2
10.1.0.4
Tenant 2 VM 2
10.1.16.12
Tenant 2 VM 3 10.1.16.21
Tenant 1 VM 3
10.1.16.47
Tenant 1 VM 4
10.1.16.85
Public Internet
10.1.0.1
Public IP address 65.37.141.1165.37.141.2465.37.141.3665.37.141.80
Load Balancer
L3 Core
Pod 1 L2 Switch
Pod 3 L2 Switch
10.1.16.1
…
…10.1.8.1Pod 2 L2 Switch
1 Firewall per Virtual Machine
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
…VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
VMVM
VM…
A Million Firewalls?
Networking Mode: Advanced• Network virtualization
– Networks can have the same subnet range– Routing, ACL between networks– Services provided at the edge
• NAT, Firewall, LB, VPN, etc
Virtual Network AppliancesNetwork services are often provided by virtual appliances.These are either commercial appliances in the virtual form factor or Linux-based networking appliances
Virtual Network NicPublic Network Nic
Control Network Nic
Virtual Router
Multi-tier virtual networking
VLA
N 2
724
DB VM 1
Web VM 1
Web VM 3
Web VM 2
VLA
N 1
01
App VM 1
App VM 2
VLA
N 3
98
VR
Internet
CustomerPremises
IPSec VPN
Private GatewayLoadbalancer
(HW or Virtual)
Network Services• IPAM• DNS• LB [intra]• S-2-S VPN• Static Routes• ACLs• NAT, PF• FW [ingress & egress]
Virtual networking with overlays
GR
E K
EY 2
724
DB VM 1
Web VM 1
Web VM 3
Web VM 2
GR
E K
EY 1
01
App VM 1
App VM 2
GR
E K
EY 39
8
VR + vSwitches
Internet
CustomerPremises
IPSec VPN
Private GatewayLoadbalancer (Virtual)
Network Services• IPAM• DNS• LB [intra]• S-2-S VPN• Static Routes• ACLs• NAT, PF• FW [ingress & egress]
Network Offerings• Cloud users are not exposed to the nature of the service
provider
• Cloud operator designs a service catalog and offers them to end users.– Gold = {LB + FW, using virtual appliances}
– Platinum = {LB + FW + VPN, using hardware appliances}
– Silver = {FW using virtual appliances, 10Mbps}
Example: Network Service offering
CLOUDSTACK ARCHITECTURE
CloudStack Architecture
Orchestration Engine
PluginFramework
Hypervisor PluginsHypervisor Plugins
Network PluginsNetwork Plugins
Allocator PluginsStorage Plugins
APIAPI
API
StorageResource
Physical Resources
StorageResource
NetworkResourceNetwork
Resource
HypervisorResourceHypervisor
Resource
Allocator PluginsAllocatorPlugins
1 2
3
45
6
7
8
9
Orchestration steps usually executed in sequence
Plugin interaction
Orchestration Engine
PluginFramework Network
PluginsNetwork Plugins
APIAPIAPI
NetworkResource
NetworkResource1 2
CloudStack DB
Desired State3
Desired State
4
Async Job Mgr
Operational StateDesired State
5
6
7
8
Idempotent Idempotent
Plugin should not update CloudStack objects
Plugin Interaction Details• Resource calls are expected to be idempotent• Plugins should not update CloudStack
resources • Plugins can have their own tables inside the
CloudStack DB• No automatic re-tries