introduction to apache cloudstack by david nalley
DESCRIPTION
Apache CloudStack is a mature, easy to deploy IaaS platform. That doesn't mean that it can be done without thought or preparation. Learn how CloudStack can be most efficiently deployed, and the problems to avoid in the process. About David Nalley David is a recovering sysadmin with a decade of experience. He’s a committer on the Apache CloudStack (incubating) project, a contributor to the Fedora Project and the Vice President of Infrastructure at the Apache Software Foundation.TRANSCRIPT
#whoami
Recovering sysadmin
Apache CloudStack and Apache jclouds
Work for Citrix Open Source Business Office
VP Infra, Apache Software Foundation
Intro to Apache CloudStack
Questions
Intro to Apache CloudStack
A brief history of CloudStack
2008-2010 – Stealth mode startup (VMOps)
2009 – First production in public cloud
2010 – Released as open source (GPLv3)
2011 – Cloud.com acquired by Citrix
2012 – Transferred to the ASF (ASLv2)
2013 – Graduated as TLP
Intro to Apache CloudStack
Cloud
Infrastructure as a Service
• Compute, Network, Storage
Cloud requires:
• On-demand
• Self-service
• Scalable
• Measurable
Intro to Apache CloudStack
What is CloudStack
Orchestration platform for Infrastructure as a
Service
Focused heavily on compute
Somewhat monolithic – by design
Written in Java
Intro to Apache CloudStack
Where do we dive in?
Intro to Apache CloudStack
IaaS is Orchestration
IaaS doesn’t really provide new services, it
orchestrates existing services
•Hypervisors
•Storage
•Network
•Isolation
•Accounting
Intro to Apache CloudStack
CloudStack Hypervisor Support
KVM
XenServer
VMware vSphere
Hyper-V
Oracle VM (reappearing in 4.5)
LXC
Baremetal
Intro to Apache CloudStack
CloudStack Storage - Primary
Local
NFS
iSCSI
Distributed (GlusterFS, Ceph, etc)
$anything_hypervisor_supports
Intro to Apache CloudStack
CloudStack Storage - Secondary
NFS
Object Storage (S3 or Swift API)
Intro to Apache CloudStack
Networking
The bane of any cloud operators existence
Intro to Apache CloudStack
Networking - Challenges
Network guys are luddites (generally)
We have scale issues
Intro to Apache CloudStack
Scale
VLANs – don’t scale
Firewalls – don’t scale
Loadbalancers – don’t scale
Intro to Apache CloudStack
How we scale the network
Amazon’s Security Groups
In CloudStack we reuse the Security Groups
terminology.
SDN
The interesting pieces aren’t in Layer 2-3,
but 4-7, and still being developed.
Intro to Apache CloudStack
Resources, hierarchy, failure
domains
Regions
Zones (DCs)
Pods
Clusters
Machines
Intro to Apache CloudStack
Actual use
UI is beautiful
…..but…..
API is where real people do work.
Intro to Apache CloudStack
API
Native CloudStack API
http://cloudstack.apache.org/docs/api/
EC2, GCE APIs
https://github.com/imduffy15/ec2stack
https://github.com/NOPping/gstack
Intro to Apache CloudStack
API
Use an abstraction library (jclouds, fog,
libcloud)
Use tools that use abstraction libraries
(knife-cloudstack,
Intro to Apache CloudStack
Stop thinking of VMs
{
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
}
]
}
{
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
knife cs stack create hadoop_cluster_a
Prereqs
Configuration management
Automated provisioning
Monitoring
Intro to Apache CloudStack
Jevon’s Paradox
Intro to Apache CloudStack
Rants
Portability is a red herring – Config
Management is a prereq
Portability is pointless – moving data is the
problem
Scale is hard – expect to iterate, a lot.
Failure is even harder – fail proactively
Intro to Apache CloudStack
Next gen predictions
Lots of people need IaaS
Many people need workload orchestration
(Kubernetes, Apache Mesos, etc)
Intro to Apache CloudStack
Questions
Intro to Apache CloudStack
Get Involved
Web: http://cloudstack.apache.org/
Mailing Lists: cloudstack.apache.org/mailing-lists.html
IRC: irc.freenode.net: 6667 #cloudstack
Twitter: @cloudstack
LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859
If it didn’t happen on the mailing list, it didn’t happen.
Intro to Apache CloudStack