CloudStack Architecture QQ: : Cloudstack : Whats Cloudstack? IaaS Infrastructure as a service PaaS SaaS Cloud Compute Amazon Rackspace Joyent HP cloud TATA communications Openstack Cloudstack Eucalyptus OpenNelula VMware vCloud Director Abiquo s ABC Cloud.com, 2011 Citrix Sheng Liang IaaS Java 2012 4 Apache ASF Apache License v2 Cloudstack KT IDC Frontier TATA Ninefold Supports Multiple Cloud Strategies Multi-tenant Public Cloud Dedicated resources Security & total control Internal network Managed by Enterprise or 3 rd party Mix of shared and dedicated resources Elastic scaling Pay as you go Public internet, VPN access Hosted Enterprise Cloud Dedicated resources Security SLA bound 3rd party owned and operated Private CloudsPublic Clouds On-premise Enterprise Cloud Compute NetworkStorage Admin Users Org A Admin Users Org B Users Primary Storage Secondary Storage Architecture Overview Hosts Servers onto which services will be provisioned Primary Storage VM storage Cluster A grouping of hosts and their associated storage Pod Collection of clusters Network Logical network associated with service offerings Secondary Storage Template, snapshot and ISO storage Zone Collection of pods, network offerings and secondary storage Management Server Farm Responsible for all management and provisioning tasks Core CloudStack Components Zone Pod Cluster Network Primary Storage Host VM Pod Cluster Secondary Storage Deployment Architecture Pod 1 . Cluster N Access Layer Host 2 Cluster 1 Host 1 Host Hypervisor Cluster Host Cluster Host Primary Storage Cluster Pod Pod 2 Availability Zone Pod Secondary Storage Availability Zone Primary Storage Zone 1 . L3 switch Secondary Storage Pod N Mgmt Server Internet Single Management Server can manage multiple zones Zones can be geographically distributed but low latency links are expected for better performance Single MS node can manage up to 5K hosts. Multiple MS nodes can be deployed as cluster for scale or redundancy Mgmt Server Mgmt Server MySQL DB Back Up DB User API Admin API Load Balancer Mgmt Server Mgmt Server Mgmt Server Mgmt Server Mgmt Server Mgmt Server MySQL DB User API Admin API Replication Management Server Mgmt Server Mgmt Server Mgmt Server 5K Host Mgmt Server Hypervisor Support Management Server Interaction with Hypervisors Managem ent Server XenServer ESX vCenter KVM Agent OVM Agent XAPIHTTP XS 5.6, 5.6FP1, 5.6 SP2, 6.0 Incremental Snapshots VHD NFS, iSCSI, FC & Local disk Storage over-provisioning: NFS XS 5.6, 5.6FP1, 5.6 SP2, 6.0 Incremental Snapshots VHD NFS, iSCSI, FC & Local disk Storage over-provisioning: NFS ESX 4.1, 5.0 Full Snapshots VMDK NFS, iSCSI, FC & Local disk Storage over-provisioning: NFS, iSCSI ESX 4.1, 5.0 Full Snapshots VMDK NFS, iSCSI, FC & Local disk Storage over-provisioning: NFS, iSCSI RHEL 6.0, 6.1, 6.2 Full Snapshots (not live) QCOW2 NFS, iSCSI & FC Storage over-provisioning: NFS RHEL 6.0, 6.1, 6.2 Full Snapshots (not live) QCOW2 NFS, iSCSI & FC Storage over-provisioning: NFS OVM 2.2 No Snapshots RAW NFS & iSCSi No storage over- provisioning OVM 2.2 No Snapshots RAW NFS & iSCSi No storage over- provisioning Multi-tenancy Multi-tenancy & Account Management Domain is a unit of isolation that represents a customer org, business unit or a reseller Domain can have arbitrary levels of sub-domains A Domain can have one or more accounts An Account represents one or more users and is the basic unit of isolation Admin can limit resources at the Account or Domain levels Admin Org A Admin Reseller A Domain Admin Org C Sub-Domain User 1 User 2 Group B Account Group A Account VMs, IPs, Snapshots Resources Virtual Machine Management VMs Lifecycle Created Stoped Running Down Destroyed Start Stop Destroy Reboot Start Restore Fail Start Users Start Stop Restart Destroy Restore Migration VM Operations Console Access CPU Utilized Network Read Network Writes VM Status Change Service Offering Change Service Offering 2 CPUs 1 GB RAM 20 GB 20 Mbps 2 CPUs 1 GB RAM 20 GB 20 Mbps 4 CPUs 4 GB RAM 200 GB 100 Mbps 4 CPUs 4 GB RAM 200 GB 100 Mbps Volume & Snapshot Management Volume VM 1 Add / Delete Volumes Add / Delete Volumes Schedule Snapshots Schedule Snapshots Hourly Daily Weekly Monthly Now Create Templates from Volumes Create Templates from Volumes VolumeTemplate View Snapshot History View Snapshot History . Storage Pod 1 Host 2 Cluster 1 Host 1 Primary Storage L3 switch Secondary Storage L2 switch Configured at Cluster-level. Close to hosts for better performance Stores all disk volumes for VMs in a cluster Cluster can have one or more primary storages Local disk, iSCSI, FC or NFS Primary Storage Configured at Zone-level Stores all Templates, ISOs and Snapshots Zone can have one or more secondary storages NFS, OpenStack Swift Secondary Storage Storage Type Support XenServervSphereKVM NFS Supported iSCSI Supported Supported via VMFS Supported via Clustered Filesystems Fiber Channel Supported via Pre- existing Supported via VMFS Supported via Clustered Filesystems Local Disk Supported Not Supported Network Router L3 Core Switch Access Layer Switches Availability Zone Servers CloudStack Mgmt Server Cluster Secondary Storage Pod 1 Pod 2 Pod 3 Pod N MySQL Load Balancer Operations Admin and Cloud API Users Physical Network Guest Traffic Virtual Router Virtual Router VM1 VM2 VM3 VM Guest Traffic ( /24) Public Traffic ( *.*) Internet Link-Local Network ( *.*) DHCP NAT Load Balancing VPN Gateway Guest Networks with L3 isolation Guest 1 VM 1 Guest 2 VM 1 Guest 1 VM 2 Guest 2 VM 2 Public Internet Public IP address Guest address Guest address Guest address Guest address Load Balancer Guest 2 VM 3 Guest 1 VM 3 Guest 1 VM 4 Guest address Guest address Guest address L3 Core Switch Pod 1 L2 Switch Pod 3 L2 Switch Pod 2 L2 Switch Security Groups Ingress/egress Ingress egress Account SG SG DB Security Group Web Security Group Web VM DB VM Web VM DB VM Web VM System VMs Whats System VMs? Stateless, can be destroyed and recreated from database state Highly Available Communicates with Management Server over management network Usually have 3 interfaces: control, guest and public Virtual Router, Console Proxy VM, Secondary Storage VM Virtual Router VM Provides multiple network services IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN User-data, Meta-data, SSH keys and password change server Redundancy via VRRP MS configures VR over SSH Proxied via the hypervisor on XS and KVM Console Proxy VM Provides AJAX-style HTTP-only console viewer Grabs VNC output from hypervisor Scales out (more spawned) as load increases Java-based server Communicates with MS over message bus Secondary Storage VM Provides image (template) management services Download from HTTP file share or Swift Copy between zones Scale out to handle multiple NFS mounts Java-based server communicates with MS over message bus Usage & API Usage listUsageRecords Usage Cloud-Management Cloud-Usage Billing listUsageRecords API REST Architecture Web Services Support POST/GET requests Return XML or JSON response formats Root Admin, Domain Admin, User