fossetcon: crash course on open source cloud computing

Download Fossetcon: Crash Course on Open Source Cloud Computing

Post on 22-Jan-2015




4 download

Embed Size (px)


This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment. Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies. [Updated with new Docker projects]


  • 1. Crash Course In Open SourceCloud ComputingMark HinkleSenior Director, Open Source SolutionsCitrix updated: 9/11/2014

2. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comABOUT MEI Help Build Open Source EcosystemsOpen Source Experience Manage Citrix Open Source Business Office Apache CloudStack Committer and PMC Member Advisory boards Gluster and Xen Project Joined Citrix via acquisition July 2011 Zenoss Core open source project to 100,000 users,1.5 million downloads Former LinuxWorld Magazine Editor-in-Chief Open Management Consortium organizer Author - Windows to Linux Business DesktopMigration Thomson NetDirector Project - Open Source ConfigurationManagementFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 3. You must give appropriate credit, provide a link to the license, and indicate if changeswere made. You may do so in any reasonable manner, but not in any way that suggests the licensorendorses you or your use.ShareAlike If you remix, transform, or build upon the material, you must distribute your contributionsunder the same license as the original.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comSlides Available on Slideshare:Creative Commons Attributions-ShareAlike 4.0 InternationalShare copy and redistribute the material in any medium or formatAdapt remix, transform, and build upon the materialfor any purpose, even commercially.The licensor cannot revoke these freedoms as long as you follow the license terms.FOSSETCON 2014 - Crash Course in Open Source Cloud Computing 4. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comAGENDA Vetting Open Source Cloud Projects What is Cloud in 60 Seconds Virtualization Infrastructure-as-a-Service Platform-as-a-Service SDN Open Source for the Amazon Web ServicesFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 5. Code Velocity Committers Committer Reputation User-driven or Vendor-DrivenInnovation User Activity Corporate Support* Reputation of Foundation*By Mark R. Hinkle@mrhinklemrhinkle@gmail.comVETTING OPEN SOURCEHPowR caOn yJouE telCl if TtheSyre LegitFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 6. http://activity.openstack.orgBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comOPEN SOURCE ANALYSISVisualizing Community ActivityFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 7. By Mark R. Hinkle@mrhinklemrhinkle@gmail.com60 SECOND CLOUD DEFINITIONJust because Software Marketing Guys Think its the Internet5 CHARACTERISTICS OF CLOUD1. On-Demand Self-Service2. Broad Network Access3. Resource Pooling4. Rapid Elasticity5. Measured ServiceUser Cloud a.k.a.SOFTWARE-AS-A-SERVICEDeveloper Cloud a.k.a.PLATFORM-AS-A-SERVICESystems Cloud a.k.a.INFRASTRUCTURE-AS-A-SERVICEFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 8. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comSCALE-UP SCALE OUTElasticity and the cloudVertical Scaling (Scale-Up)Allocate additional resources toVMs, requires a reboot, no need fordistributed app logic, single-point ofOS failureHorizontal Scaling (Scale-Out)Application needs logic to work indistributed fashion (e.g. HA-Proxyand Apache Hadoop)FOSSETCON 2014 - Crash Course in Open Source Cloud Computing 9. HYPERVISORS AND CONTAINERSDifferences in virtualizationBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comType 1 HypervisorsVMware, Xen Project, Hyper-VType 2 HypervisorsKVM, VirtualBoxFOSSETCON 2014 - Crash Course in Open Source Cloud ComputingContainersLXC 10. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comVIRTUALIZATIONCarving up compute resourcesOPEN SOURCE Xen Project Citrix XenServer KVM VirtualBox OpenVZ LXCPROPRIETARY VMware Microsoft Hyper-V OracleVM (Based on Xen Project)FOSSETCON 2014 - Crash Course in Open Source Cloud Computing 11. OPEN VIRTUALIZATION FORMATSVirtualization PayloadsFormats for hypervisors/cloudtechnologies: Amazon - AMI KVM QCOW2 VMware VMDK Xen Project IMG Hyper-V - VHD Virtual Hard Disk LXC local file system/mount point -By Mark R. Hinkle@mrhinklemrhinkle@gmail.comOpen VirtualizationFormat (OVF) is anopen standard forpackaging anddistributing virtualappliances or moregenerally software tobe run in virtualmachines.Docker*FOSSETCON 2014 - Crash Course in Open Source Cloud Computing 12. Lets your run a Linux system within A container is a group of processes on aLinux box, put together the provide anisolated environment From the inside, it looks like a VM Externally it looks like normal processes chroot on steroidsBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comLINUX CONTAINERS (LXC)Lightweight Linux Virtualizationanother Linux systemFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 13. Different file formats for virtual machines VMware uses vmdk file format, Xen and Hyper-V use VHD, KVM uses Raw or QCOW2 Guest images may be processor architecture VMware and Xen can manage SCSI devices, but KVM and Xen can use virtio drivers but not VMware uses a proprietary agent inside theguest OS (VMware tools) which does not workwith Xen or KVM Xen uses VirtIo and ParaVirtualized drivers, XenBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comTHE PORTABILITY PROBLEMContainers compared to Hardware VirtualizationboundKVM cannotVMwareusesFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 14. Code Application is stored Build Code is built (Jenkins) Test Unit tests areBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comCONTINUOUS INTEGRATIONRebuild Applications on any Cloud and/or Virtualized Infrastructurein a repository(Subversion,Git)automated (Jenkins) Deploy Deploy code toserver various waysCodeBuildTestDeployThoughtworks Go Open SourceContinuous Deliver SystemFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 15. PACKER MULTIPLATFORM VMCREATIONPacker is easy to use and automates thecreation of any type of machine image. Itembraces modern configurationmanagement by encouraging you to useautomated scripts to install and configurethe software within your Packer-madeimages.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comTo learn more please visit:www.packer.ioOpen source Automation for VMsFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 16. DOCKER CONTAINER PACKAGINGOpen source LXC Packaging EngineDocker is an open-source project to easilycreate lightweight, portable, self-sufficientcontainers from any application. The samecontainer that a developer builds and testson a laptop can run at scale, in production,on VMs, bare metal, public clouds andmore.By Mark R. Hinkle@mrhinklemrhinkle@gmail.comTo learn more please visit:www.docker.ioFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 17. Compliment to LXC not a replacement Managed daemonized processes on Linux Create ability to re-use and manage similar Content agnostic Hardware agnostic Easy to automate Integrated with other tools: Chef, OpenShift,By Mark R. Hinkle@mrhinklemrhinkle@gmail.comWHAT IS DOCKERSystem for Managing and Deploying LXC Containersusing LXCapplicationsPuppet, VMware, etc.FOSSETCON 2014 - Crash Course in Open Source Cloud Computing 18. DOCKERS GROWINGECOSYSTEMBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 19. Kubernetes builds on top of Docker toconstruct a clustered container schedulingservice. Kubernetes enables users to aska cluster to run a set of containers. Thesystem will automatically pick workernodes to run those containers on, whichwe think of more as "scheduling" than"orchestrationTo learn more please visit: Greek for ShipmasterBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comKUBERNETESContainer Cluster Management SchedulerFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 20. DOCKER RELATEDPROJECTS Fig -Fast, isolated development environments Flynn - Next-generation application platform Panamax Drag-and-Drop Docker Containerization Project Atomic JEOS designed to run Dockercontainers Weave The Docker Network 13,000+ Docker-related repos on GithubBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comFOSSETCON 2014 - Crash Course in Open Source Cloud Computing 21. Apache Mesos is a cluster manager that simplifies thecomplexity of running applications on a shared pool ofservers. Largely supported by Twitter, used by LinkedIn,AirBNB too.Features Fault-tolerant replicated master using ZooKeeper Scalability to 10,000s of nodes Isolation between tasks with Linux Containers Multi-resource scheduling (memory and CPU aware) Java, Python and C++ APIs for developing newBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comAPACHE MESOSOne to many tools for managing large numbers of devicesparallel applications Web UI for viewing cluster stateTo learn more please visit: 2014 - Crash Course in Open Source Cloud Computing 22. ZooKeeper is a centralized service formaintaining configuration information,naming, providing distributedsynchronization, and providing groupservices. All of these kinds of servicesare used in some form or another bydistributed applicationsBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comAPACHE ZOOKEEPERCentralized Server to Service Distributed AppsTo learn more please visit: 2014 - Crash Course in Open Source Cloud Computing22 23. INFRASTRUCTURE-AS-A-SERVICECompute OrchestrationProject Year Started License VirtualizationBy Mark R. Hinkle@mrhinklemrhinkle@gmail.comFOSSETCON 2014 - Crash Course in Open Source Cloud ComputingTechnologiesApacheCloudStack2008 Apache (Bare Metal), Xenserver,KVM, LXC VMware Hyper-VEucalyptus 2006 GPL Xen, KVM, VMware(commercial version)OpenNebula 2005 Apache Xen, KVM, VMwareOpenStack 2010 (Developed byNASA by Anso Labspreviously)Apache VMware ESX and ESXi, ,Xen, XenServer, KVM,LXC, QEMU and VirtualBox 24. By Mark R. Hinkle@mrhinklemrhinkle@gmail.comOPENSTACKThe Boy Band of the Open Source C


View more >