openstack cookbook: freedom in the cloud

Christopher J. Anderson 9/11/12

OpenStack CookBook

Freedom in the Cloud…

By Chris Anderson

OpenStack CookBook Freedom in the Cloud…

-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


Topics of Discussion o What is OpenStack

• Who is behind OpenStack? • Who can use OpenStack? • Why make it open source?

The Mission 4 Founding Principles

• Open Source • Open Design • Open Development • Open Community

o Why OpenStack?

Control & Flexibility Industry Standard Proven Software Compatible & connected

o OpenStack Release Schedule

Austin Bexar Cactus Diablo Essex

o OpenStack Software Projects

COMPUTE (aka Nova) OBJECT STORAGE (aka Swift) IMAGE SERVICE (aka Glance)

o NEW - OpenStack Software Projects

IDENTITY (aka Keystone) DASHBOARD (aka Horizon)


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


o OpenStack Community Projects Atlas-LB Burrow Clanavi CloudGateway Crowbar Dodai Donabe Juju Lunr Melange MultiClusterZones Orchestra Quantum RedDwarf Topology Ubuntu Cloud Live

References http://wiki.openstack.org/ http://docs.openstack.org/ http://nova.openstack.org/ http://swift.openstack.org/ http://glance.openstack.org/ https://launchpad.net/nova


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


What is OpenStack OpenStack is a global collaboration of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. The project aims to deliver solutions for all types of clouds by being simple to implement, massively scalable, and feature rich. The technology consists of a series of interrelated projects delivering various components for a cloud infrastructure solution. Who is behind OpenStack? -‐ Founded by Rackspace Hosting and NASA, OpenStack has

grown to be a global software community of developers collaborating on a standard and massively scalable open source cloud operating system. Our mission is to enable any organization to create and offer cloud computing services running on standard hardware.

Who can use OpenStack? -‐ Corporations, service providers, VARS, SMBs,

researchers, and global data centers looking to deploy large-scale cloud deployments for private or public clouds leveraging the support and resulting technology of a global open source community.

Why make it open source? -‐ All of the code for OpenStack is freely available under the

Apache 2.0 license. Anyone can run it, build on it, or submit changes back to the project. We strongly believe that an open development model is the only way to foster badly-needed cloud standards, remove the fear of proprietary lock-in for cloud customers, and create a large ecosystem that spans cloud providers.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


The Mission

"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private

cloud providers regardless of size, by being simple to implement and massively scalable."

------------------------------------------------------------------------------------------------------------

4 Founding Principles Open Source: -‐ No licensed (pay for) versions. All code is released

under the Apache 2.0 License allowing the community to use it freely.

Open Design: -‐ Every 6 months the development community will

hold a design summit to gather requirements and write specifications for the upcoming release.

Open Development: -‐ Publicly available source code repository is

available on LAUNCHPAD & GITHUB throughout the development process.

Open Community: -‐ Our core goal is to produce a healthy, vibrant

development and user community. All processes will always be documented, open, & transparent.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


Why OpenStack? Control and Flexibility

-‐ Open source platform means you’re never locked to a

proprietary vendor, and modular design means you can integrate with legacy or third-party technologies to meet your business needs.

-‐ Hypervisor support for Microsoft Hyper-V, Citrix XenServer, Xen, KVM, VMWare ESX, LXC, QEMU, and UML.

Industry Standard

-‐ More than 60 leading companies from over a dozen

countries are participating in OpenStack, including Cisco, Citrix, Dell, Intel and Microsoft, and new OpenStack clouds are coming online across the globe.

Proven Software

-‐ Running of OpenStack cloud means running the same

software that today powers some of the largest public and private clouds in the world.

Compatible and Connected

-‐ Compatibility with public OpenStack clouds means

enterprises are prepared for the future—making it easy to migrate data and applications to public clouds when conditions are right—based on security policies, economics, and other key business criteria.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


OpenStack Release Schedule Starting with the Diablo release, the OpenStack release cycle abandoned its 3-month time-based cycle for a coordinated 6-month release cycle with frequent development milestones.

These codenames are chosen after a city or county near where the corresponding OpenStack design summit took place:

Austin: Oct, 2010 o Hypervisor Support: Xen, KVM, QEMU, User Mode Linux

Support o Security Groups Implementation o Rescue Mode Experimental - Glance (Image Store as a

Service) Bexar: Feb 3, 2011 o OpenStack Compute ready for enterprise private cloud

deployments and mid-size service provider deployments o Enhanced documentation o Easier to install and deploy Cactus: April 15, 2011 o OpenStack Compute ready for large service provider

scale deployments Diablo: (4th version – Sept, 2011) o Planning (Design, Discuss & Target) o Implementation (Milestone iterations) o QA (The road to full release) o Intended to be ‘Rackspace ready’ release Essex: (5th Version – April 5th, 2012) o Focuses on quality, usability & extensibility across

Enterprise, service providers & (HPC) high performance computing deployments.

o Leverage pools of on-demand, self-managed compute, storage & networking resources to build efficient, automated private & public cloud infrastructure.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


OpenStack Software Projects OpenStack is a collaborative software project designed to create freely available code, badly needed standards, and common ground for the benefit of both cloud providers and cloud customers. OpenStack is currently three projects: ------------------------------------------------------------------------------------------------------------

COMPUTE (aka Nova) Open source software and standards for large-scale deployments of automatically provisioned virtual compute instances.

OpenStack Compute is open source software designed

to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. It gives you the software, control panels, and APIs required to orchestrate a cloud, including running instances, managing networks, and controlling access through users and projects. OpenStack Compute strives to be both hardware and hypervisor agnostic, currently supporting a variety of standard hardware configurations and seven major hypervisors. Features: http://www.openstack.org/software/openstack-compute/ • Component based architecture: Quickly add new roles. • Highly available: Scale to very serious workloads • Fault-Tolerant: Isolated processes avoid cascading failures • Recoverable: Failures should be easy to diagnose, debug,

and rectify. But no centralized logging. • Open Standards: Be a reference implementation for a

community-driven api • API Compatibility: Nova strives to provide API-

compatible with popular systems like Amazon EC2 • Advancement of OpenStack API as industry standard.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


OBJECT STORAGE (aka Swift) Open source software and standards for large-scale, redundant storage of static objects.

Swift is open source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. It is not a file system or real-time data storage system, but rather a long-term storage system for a more permanent type of static data that can be retrieved, leveraged, and then updated if necessary. Primary examples of data that best fit this type of storage model are virtual machine images, photo storage, email storage and backup archiving. Having no central "brain" or master point of control provides greater scalability, redundancy and permanence. http://www.openstack.org/software/openstack-storage/

------------------------------------------------------------------------------------------------------------


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


IMAGE SERVICE (aka Glance) Provides discovery, registration, and delivery services for virtual disk images.

The Glance project provides services for discovering,

registering, and retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image. Similar to Amazon’s AMI Marketplace. Virtual disk images can be stored in a variety of back-end stores, including OpenStack Object Storage, as well as local disks. Clients can register new virtual disk images with the Image Service, query for information on publicly available disk images, and use the Image Service's client library for streaming virtual disk images.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


NEW - OpenStack Software Projects In addition to the three existing core projects above, two new projects were incubated with the Diablo release and will be promoted to ‘core’ for the forthcoming Essex version. ------------------------------------------------------------------------------------------------------------

IDENTITY (aka Keystone) http://www.openstack.org/software/openstack-shared-services/

Code-named Keystone, the OpenStack Identity Service provides unified authentication across all OpenStack projects and integrates with existing authentication systems. ------------------------------------------------------------------------------------------------------------

DASHBOARD (aka Horizon) http://www.openstack.org/software/openstack-dashboard/

Code-named Horizon, the OpenStack Dashboard enables administrators and users to access and provision cloud-based resources through a self-service portal.


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


OpenStack Community Projects These projects are provided by the wider community and have no official endorsement. They are projects that are related to one of more OpenStack projects and may extend the functionality or usability of the core OpenStack projects. ------------------------------------------------------------------------------------------------------------

Atlas (LBaaS) Project Leads: Youcef Laribi and Uma Goring Overview: The Atlas project will provide Load Balancing as a Service or LBaaS for short. The goal is for load balancing vendors to be able to write adapters to this service for their technology. -----------------------------------------------------

Burrow Project Lead: Eric Day Overview: A messaging queue system for multi-tenant cloud use. Burrow has a simple API and modular design. -----------------------------------------------------

Clanavi Project Lead: Yas Naoi Overview: Cloud management tool based on Drupal moduls for supporting multiple clouds. Supported cloud functionalities include server templates, clusters, monitoring, billing, resource allocation, reliability, scalability and more. -----------------------------------------------------


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


CloudGateway Project Lead: Shivan Bindal Overview: A common interface used to manage multiple clouds based on the RightScale gateway. Ultimately, the Cloud Gateway should evolve into an interoperable cloud standard. -----------------------------------------------------

Crowbar Project Lead: Rob Hirschfeld (https://github.com/dellcloudedge/crowbar) Overview: A DevOps inspired cloud installation and maintenance system that allows users to quickly deploy a fully functioning OpenStack cloud. The code design is modular so that operators can choose which components, infrastructure, and vendors they want to include in their system; however, modules are also system aware and cross reference each other to build an integrated operating environment. Crowbar is a wrapper for the open source Opscode Chef Server (which is installed as part of package) with the addition of PXE/DHCP based node discovery, an orchestration state machine, and system level deployment semantics. Crowbar provides a foundation for adapting to software changes while creating scalable and repeatable cloud deployments. -----------------------------------------------------

Dodai Project Lead: Shigetoshi Yokoyama; (https://github.com/nii-cloud/dodai) Overview: "Cluster as a Service" Managing multiple clusters for openstack clouds, Hadoop, and other diverse frameworks. Clusters of commodity servers are used for a variety of distributed applications like simulation, data


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


analysis, web services, and so on. No single framework can fit every distributed application. Users can get clusters just by specifying the configuration of them. Dodai has sub-projects like Dodai-deploy and Dodai-compute. -----------------------------------------------------

Donabe Project Lead: Rick Clark Overview: Donabe is a container service, a group of resources created and/or managed as one unit, with an initial focus on network containers. -----------------------------------------------------

Juju Project Lead: Gustavo Niemeyer (http://juju.ubuntu.com) Overview: Juju focuses on managing the service units you need to deliver a single solution, above simply configuring the machines or cloud instances needed to run them. Charms developed, tested, and deployed on your own hardware will operate the same in OpenStack, or any other EC2 API compatible cloud. Furthermore, juju allows you to test your deployments locally via LXC containers, providing an inexpensive and fast way to test your OpenStack deployment or any other scale-out, multi-node solution. Through the use of charms, juju provides you with shareable, re-usable, and repeatable expressions of DevOps best practices. You can use them unmodified, or easily change and connect them to fit your needs. Deploying a charm is similar to installing a package on Ubuntu: ask for it and it’s there, remove it and it’s completely gone. -----------------------------------------------------


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


Lunr Project Lead: Chuck Thier Overview: An open commodity storage platform that will integrate with the Nova Volume service. -----------------------------------------------------

Melange Project Lead: Troy Toman Overview: Melange will provide network information services with a focus on IP address management and address discovery. Melange will be a standalone service with its own API but fully integrate-able with Nova. -----------------------------------------------------

MultiClusterZones Project Lead: Sandy Walsh Overview: A Nova deployment is called a Zone. Internal deployment nuances, such as hostnames and service information, are hidden to users outside of a Zone. Zones may be joined together to form a hierarchy of OpenStack services. This may be used to partition OpenStack into geographical regions or business units. A Zone may have the full suite of Nova services or can be as simple as the API & Scheduler services. The Distributed Scheduler provisions servers across Zones. -----------------------------------------------------


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


Orchestra Project Lead: Dave Walker (http://wiki.ubuntu.com/Orchestra) Overview: A collection of what Ubuntu feels are the best free software services for provisioning, deploying, hosting, managing, and orchestrating enterprise data center infrastructure services, by, with, and for Ubuntu Server. Orchestra, via Juju, enables users to quickly deploy an OpenStack cloud solution across any x86/x86_64 (and soon ARM!) hardware in the datacentre. Using other leading tools, Orchestra also monitors server activity centrally and provisions new server resources rapidly as and when they are needed. In short, Orchestra allows organisations to provision clouds in minutes, and manage and maintain it using commonly accepted and leading opensource tools. -----------------------------------------------------

Quantum Project Lead: Dan Wendlandt Overview: A service providing network connectivity-as-a-service for devices managed by other OpenStack services. It exposes a generic and extensible API, allowing users to build and manage their networks, and uses a pluggable architecture, thus enabling different technologies to implement the logical abstractions exposed by the API. -----------------------------------------------------

RedDwarf Project Lead: Michael Basnight Overview: A scalable relational database service that allows users to quickly and easily utilize the features of a relational database without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database


-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐


instances as needed. Initially, the service will focus on providing resource isolation at high performance while automating complex administrative tasks including deployment, configuration, patching, backups, restores, and monitoring. -----------------------------------------------------

Topology Project Lead: Eldar Nugaev Overview: A topology service which is being augmented by a number of adapters as a primary way to provide a solution for failure zones centered IaaS deployment. -----------------------------------------------------

Ubuntu Cloud Live Project Lead: Ante Karamatić (https://launchpad.net/cloud-live) Overview: The project uses components of OpenStack to deliver a fully functional development/demo cloud for use on your laptop or desktop. By leveraging an Ubuntu live ISO image, you can safely use the cloud without harm to your own machine configuration. By providing the desktop, you even have direct access to the OpenStack Dashboard (Horizon). -----------------------------------------------------

MultiClusterZones Project Lead: Sandy Walsh Overview: A Nova deployment is called a Zone. Internal deployment nuances, such as hostnames and service ------------------------------------------------------------------------------------------------------------

openstack cookbook: freedom in the cloud

Documents

sandy walsh overview

internal deployment nuances

virtual disk images

openstack release schedule

open source software

openstack compute ready

openstack software projects

openstack community projects