OpenStack Meetup San Antonio TXOct. 7th, 2014

Twitter: Meetup:

@SAOpenStackers#SAOpenStack

www.meetup.com/SA-Open-Stackers

Thank you to our sponsors:

OpenStack Meetup San Antonio TXOct. 7th, 2014

Who am I?

• eCommerce Startup• Built and Sold in 2007

• Rackspace Hosting• Enterprise Sales• Rackspace CloudBuilders

• Canonical (Ubuntu)• Helped customers design and deploy

Ubuntu OpenStack

• Cloud Consultant• Help clients design, operationalize,

automate and productize public & private clouds.

A little more about me:

OpenStack 101

Agenda

Module 1OpenStack Overview

Module 2OpenStack Architecture

Module 3Installing OpenStack

•Purpose, History, and Contributions

•Why OpenStack  

• Architecture Overview• Projects Definitions• Codebase

• DevStack• Distros (Linux)• StackForge

MODULE 1

OpenStack™ Overview

How would you define it?

Nebulous

Ethereal

Foggy

Platform as a Service (PaaS)

Infrastructure as a Service (IaaS)

Software as a Service(SaaS)

Broad Network Access

Rapid Elasticity

Resource PoolingOn-DemandSelf Service

Cloud Defined

EssentialCharacteristics

Service

Models

Deployment

Models Public Hybrid Community Private

Measured Service

NIST Cloud Model

Why Cloud?

• Reduce overhead on IT• Accelerate development, enable Dev/Ops workflows• Build against new development paradigms• Enable Application deployment mobility

• Cloud is not necessarily the right answer for:• Enterprise apps built on very specific non-reproducible infrastructure• Application that are built on “infrastructure resiliency” vs. “application resiliency” models

• Cloud != Virtual Managed Hosting• Cloud == self-service infrastructure and services

Amazon Web Services

• Scalable cloud architecture• Programmable infrastructure• Self-service consumption

model• Cost-efficient infrastructure

solution

Why Cloud?

Why OpenStack for Customers?

1. Open: No vendor lock-in

2. Platform: Solution for private and public clouds

3. Cost: Low software costs, automation reduces CapEx

4. Storage: Low-cost storage solutions – Ceph, Swift, Cinder

5. Flexibility: Modular software architecture

DevOps – Development with Operations

• Agile/Extreme/Lean/Etc. application development expect rapid turn from test develop production

• Model for Deployment built into the test/development lifecycle

• Unit test• Continuous Integration• Move from semi-annual release to

daily or weekly releases• Some iterate ~40x/day dev

production!

User Shift to Self Service

• Users of a public, private, and hybrid cloud all like having the on-demand option of deploying applications

• This typically is modeled after most public cloud operations where the user can simply select an application from a catalog and have it deployed instantly

• Lifecycle management, chargeback, and accounting need to be tied into this as well

What can I do now?

• Iterate on your application on a daily if not hourly basis from dev->test->production

• Deploy your application to multiple locations, with the same management toolsets

• Manage resources on demand, rather than via request/review process

• My infrastructure capacity can be programmatically scaled in real time to meet application/consumer demands

Welcome to OpenStack

The Cloud needs an Open Source platform to achieve Internet Scale:

What is OpenStack• OpenStack is an open source infrastructure and

application middleware for building private and public clouds.

• OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter.

• OpenStack is backed up by a global community of technologists, developers, researchers, corporations and cloud computing experts.

History

• Released July 2010• NASA Nebula (compute (NOVA))• Rackspace CloudFiles (object storage

(SWIFT))• Developer Led “Design Summits”• 6-month development to release cycle

OpenStack Foundation

• https://www.OpenStack.org/foundation/ • OpenStack Foundation, established September 2012• Elected technical committee, elected board, individual and organization membership

Companies involved…

Companies involved…

Companies involved…

OpenStackRelease Name Release Date Included ComponentsAustin 21 October 2010 Nova, Swift

Bexar 3 February 2011 Nova, Glance, Swift

Cactus 15 April 2011 Nova, Glance, Swift

Diablo (1st Production Release) 22 September 2011 Nova, Glance, Swift

Essex 5 April 2012 Nova, Glance, Swift, Horizon, Keystone

Folsom 27 September 2012 Nova, Glance, Swift, Horizon, Keystone, Quantum,Cinder

Grizzly 4 April 2013 Nova, Glance, Swift, Horizon, Keystone, Quantum,Cinder

Havana 17 October 2013 Nova, Glance, Swift, Horizon, Keystone, Neutron,Cinder, Ceilometer, Heat

Icehouse 17 April 2014 Nova, Glance, Swift, Horizon, Keystone, Neutron,Cinder, Ceilometer, Heat, Trove

Juno November 2014 Nova, Glance, Swift, Horizon, Keystone, Neutron,Cinder, Ceilometer, Heat, Trove (more tobe added)

Kilo

Project Contributions - Icehouse• Compute (Nova)• Object Storage (Swift)• Image Service (Glance)• Identity (Keystone)• Dashboard (Horizon)• Networking (Neutron)• Block Storage (Cinder)• Telemetry (Telemetry)• Orchestration (Heat)• Database Service (Trove)• Data processing (Sahara*)• Bare metal (Ironic*)• Queue service (Marconi*)• Key management (Barbican*)• DNS Services (Designate*)• Common Libraries (Oslo)

* under incubation

http://stackalytics.com/

Mellanox

iSCSI

XenAPI Storage Manager

Huawei

Sheepdog

Zadara

HP LeftHand

Storwize

SolidFire

Xenapi

SAN/HP

EMC

NetApp

11122333333445557889121315

3237

71

Other

Ceph

LVM

260

77

138

Storage Driver

Other

Chef

Puppet

86

35

72

Deployment tool

Anvil

CFEngine

FAI

Foreman

None

Ansible

Fuel

Other

StackOps

Substratum

Crowbar

SaltStack

PackStack

DevStack

1

1

1

1

1

2

2

2

2

2

8

8

16

39

Other

xen

kvm

145

50

312

Hypervisor

Docker

PowerVM

Other

OpenVZ

Bare Metal

hyperv

lxc

QEMU

xenserver

esx

1

1

4

5

13

13

21

23

23

41

Other

Centos

Ubuntu

49

49

111

OS

FreeBSD

Other

Scientific Linux

Fedora

SUSE Linux Enterprise

openSUSE

Debian

Windows

RHEL

1

1

2

3

3

3

6

9

21

Other

Linux Bridge

OpenVswitch

125

101

155

Network Driver

Arista

Custom

Mellanox

Modular

Juniper

MidoNet

PLUMgrid

NEC

Other

Ryu

Big Switch

Brocade

Hyper-V

Nicira

Cisco

2

2

2

2

3

3

3

4

4

6

7

12

15

22

38

Templated

KVS

PAM

LDAP

SQL

2

13

20

103

171

Identity Driver

IceHouse User Survey Results

Getting Started – Small Scale

• Try/Dev/Demo:• http://devstack.org/ - OpenStack for developers• http://trystack.org/ - Live OpenStack, funded by the Foundation

• RDO/Canonical/Suse/Fuel/Havate/Alamo/Etc. “All-in-One”• On your laptop (VMware Workstation/Fusion allows nested VMs)• On a single machine (Like any OS install, deploy to disk)

MODULE 2

OpenStack Architecture

Conceptual IaaS Architecture

Integration

Billing

Identity

Management

Admin API

Monitoring

Presentation

Logic (Control)

Resources

System APIUser

Dashboard

Orchestration Scheduling PolicyImage

RegistryLogging

Compute Volume Network

Orchestration API Portal/Catalog

Conceptual IaaS Architecture

Integration

Billing

Identity

Management

Admin API

Monitoring

Presentation

Logic (Control)

Resources

System APIUser

Dashboard

Orchestration Scheduling PolicyImage

RegistryLogging

Compute Volume Network

Orchestration API Portal/Catalog

?Telemetry

Keystone

OS API(s)

Telemetry

OS API(s) Horizon

All Services All Services All Services Glance Telemetry

Nova Cinder Neutron

Heat ? Horizon

Important Terms:

Host Operating System (Host). The operating system that is installed on your physical server or laptop that hosts virtual machines. This is commonly referred to as the host OS or host.

Guest Operating System (Guest). The operating system that is installed on your Virtual Machine. This virtual instance is independent of the host OS. It is commonly referred to as guest OS or guest.

Node. In this context, refers specifically to physical servers. Each OpenStack server is a node.

Control Node. Hosts the database, Keystone (Middleware), and the servers for the scope of theOpenStack deployment. It acts as the brains behind OpenStack and drives services such as authentication,database, and so on.

Compute Node. Has the required Hypervisor (ESX/Xen/KVM) and is your Virtual Machine host.

Network Node. Provides Network-as-a-Service and virtual networks for OpenStack.

Compute Node – Top Down • Virtual Machine

• Virtual “Bare Metal”• Runs a full copy of the Operating System• Runs on Hypervisor

• Hypervisor or Container• Hypervisor - Hardware access management and segregation

• ESX, KVM, Hyper-V, Xen, LPAR

• Container - Operating System level segregation of processes• Docker/LXC, Solaris containers

• Operating System and Process• Linux - Apache• Windows – IIS

• Bare Metal• x86, ARM, other processor• Memory• Local “block” storage subsystem

Storage

• Block Storage• A ‘block’ of bits, historically written on magnetic media• Depending on media allows sequential only, or random access to bits• Can be very sensitive to any disruptions, hence technologies like RAID

• File Storage• File can be as simple as a pointer to a a set of blocks with a mechanism for chaining

blocks together• File systems describe a higher order list of initial file blocks• Often include error correction, journaling, and other mechanisms intended to improve

stability

• Object• Effectively a very specialized file system type• Reduces some of the overhead of file system based storage, removing some limits• Act more like a massive database of “blob” items accessed by the “key” or object-id

Network

• Virtual networks are often abstracted via software (the virtual switch) and act the same as their physical counterparts

• Software Defined Networks are often mechanisms used to enable applications to manipulate the forwarding mechanism, providing application driven value

• Typically described in terms of ISO standard layers (L2, L3, etc.)• Enable connectivity, either at the media access layer (L2), or via a concept of a

routeable address (L3).• L4 and above is addressing within the compute domain, directing to an application

(L4) or even within the application itself (L5-7)

Example Reference Model

• Single Controller• Network on Controller• OVS + L3_agent• Cinder on LVM

• Separate Compute• Nova• Cinder• OVS + GRE Controller:

KeystoneNova

Neutron (L2/L3)CinderHorizon

Compute:Nova

Cinder(OVS)

Public/External: 192.168.1.0/24

Management: 172.16.0.0/24

Public/Float: 192.168.2.0/24

GRE tunnel(s)

Simple OpenStack Deployment Model

Nova:Nova-apiNova-schedulerNova-conductorNova-certNova-consoleauthNova-objectstoreNova-novncproxy

Nova:Nova-compute

Neutron:Neutron-serverNeutron-metadata-agentNeutron-l3-agentNeutron-dhcp-agentNeutron-plugin-*-agent

Cinder:Cinder-apiCinder-scheduler

Keystone:Keystone

Horizon:apache2- wsgi django app

Mysql:Mysql-server

RabbitMQ:Rabbitmq-server

Heat:Heat-apiHeat-api-cfnHeat-api-cloudwatchHeat-engine

Telemetry:Telemetry-agent-centralTelemetry-apiTelemetry-collector

Cinder:Cinder-volume

…

Neutron:Neutron-agent

Nova:Nova-compute

Neutron:Neutron-agent

Cinder:Cinder-volume

Linux on Bare Metal – Control serverLinux on Bare Metal with KVM Hypervisor – compute-network-storage

Linux on Bare Metal with KVM Hypervisor- Compute network

Linux on Bare Metal with Storage and iSCSI driver- Storage

OpenStack Project Overview

Dashboard Demo

Messaging

Service Interaction

Horizon (Dashboard)

• Provides graphical interfaceto user and administrators.

• Gives access, provisionand automate cloud–based resources.

• Is a modular Django web application.

• Deployed via mod_wsgiin Apache.

Horizon

Horizon Database

HTTP(S)

OpenStack Image API

OpenStack Identity API

OpenStack Network API

OpenStack Compute API

OpenStack Block Storage API

Heat

Nova (Compute)

• OpenStack Compute (Nova) is a cloud computing fabric controller.

• Provides a highly scalable management framework for virtual machines.

• Designed to manage and automate pools of compute resources.

• Supporting wide variety of virtualization technologies.

• Scale up and down the infrastructure to meet demand.

Nova (Compute)

nova-compute

nova-api(OS, EC2, Admin) nova-console

nova-cert / object store

hypervisor

nova-consoleauth

nova-scheduler nova-network

nova-volume/cinder

volume provider(iSCSI, etc)

novadatabase Queue

Network Provider

Neutron Agent

libvrt, XenAPI,etc

cinder-apicinder-scheduler

amqp

cinder-volume

OpenStack Image API

OpenStack Identity API

OpenStack Compute API vnc/vmrc

nova-conductor

Swift (Object Storage)

• OpenStack Object Storage project is codenamed as Swift

• Provides cloud storage software which makes storing and retrieving data easy

• Built for scalability, optimized for durability, availability and concurrency

• Ideal for storing unstructured data that can grow without bound

swift-proxy

account container object

account DB

container DB

object store

Client Access OpenStack Object API

OpenStack Identity API

Glance (Image Service)

• Provides an API that allows querying of VM image metadata and retrieval of the actual image

• VM images can be stored in various locations ranging from a simple file system to object storage file system like Swift

• Glance has a Component based Architecture

• Highly Available• Scalable to huge workloads

glance-api

glance-registry

glance database

OpenStackIdentity API

Storage Interface(s)

OpenStackImage API

Keystone (Identity Service)

• Proves authentication to OpenStack services.

• Deals with policy management and catalog services.

• Grant tokens for authorization, creating policies, endpoints

• Users are assigned to containers called tenants

• Tenants isolates resources and identity objects

Keystone(service & Admin APIs)

token backend(kvs, memcache)

catalog backend(kvs, sql, etc)

policy backend(rules, custom)

identity backend(kvs, pam, sql)

OpenStack Identity API

Neutron (Network)

• An OpenStack project that provides NaaS between interface devices managed by other OpenStack Services

• Provides advanced networking options which Nova could not provide

• Neutron is replacement for Nova-network.

neutron–server

Neutronagents

NeutronPlugin(s)

Neutron database

Queue

OpenStack Network API

OpenStack Identity API

Cinder (Block Storage)

• OpenStack block storage component is codenamed as Cinder.

• Designed to be used as a storage resource for OpenStack Nova project

• Cinder Manages persistent Storage.

• Virtualizes pools of block storage devices and provides end users with a self service APIs

cinder-api

cinder-volume

Cinder database

cinder-scheduler

OpenStackIdentity API

volumeprovider

(iSCSI, etc)

OpenStack Block Storage API

Telemetry (aka Ceilometer, Metering)

• Infrastructure to collect metrics within OpenStack

• Primary targets are monitoring and metering

• Should be able to share collected data with variety of customers

• Telemetry provides single point of contact for a user’s billing system

Alarm Queue

Telemetry Collector

Telemetry Agents

Database Telemetry API

Telemetry Evaluator

TelemetryNotifier

OpenStackIdentity API

Telemetry notifications

Push/Polling Inputs

Telemetry Data

Heat (AutoScale)

• Template Drive Automation• Talks to IaaS components via

APIs• Integration with Horizon

(template upload, and parameter insertion), or one of 3 API inputs

• Supports Amazon Cloud Formation templates(XML/yaml) or OpenStack HOT templates

• Autoscale enabled via integration with Telemetry project

HOT template

Heat-API

Heat-engine

Heat database

Ceilometer Alarms

OS Services Nova, Neutron,

Glance, etc.

Heat Client for API Calls

OpenStack Identity API

Latest “released” Project(s)

• Trove (IceHouse):• Open source Database as a Service( DBaaS).• Provides scalable and reliable Cloud Database.• Provisioning functionality for both relational and non-relational database engines• Goal is to allow users to quickly and easily utilize the features of a relational database.• Cloud users and database administrator can provision and complex administrative tasks

including deployment, configuration, patching, backups, restores and monitoring.

Incubated Projects (Juno?)

• Ironic:• Aims to provision bare metal machines instead of virtual machines• Forked from the Nova Bare metal driver• By default, it will use PXE and IPMI in concert to provision and turn on/off machines• Also supports vendor-specific plugins which may implement additional functionality.

• Marconi – Queue Service• RESTful multi-tenant capable queues

• Sahara – Data Processing (Big Data)• Hadoop-as-a-Service

Project Stages

• Concept – Described as a project level blueprint, example code

• Incubation – Community support to the point where OpenStack agrees to help host/manage – Expected for inclusion in the core services

• Core – Required feature to be consider a full OpenStack system

Architecture Resources:

Mirantis Arch Overview Slide Deck: http://goo.gl/RRNHGp

Official OpenStack Documentation: http://docs.openstack.org/

MODULE 3

Installing OpenStack

Installation Methodologies

• Manual• DevStack• Distros (linux)• Puppet/Chef (Stackforge)

DevStack

• Developer toolset• Single scripted interface• Flexible ‘per service’ download of code from git repositories• All logs broken out in separate screen sessions

• Not intended (or really good for) production deployments

Distro solutions

• Each distro has it’s own method of installation

• RedHat (RedHat, CentOS, Fedora)• PackStack, Scripted puppet runs

• Canonical (Ubuntu)• Juju/MaaS, Canonical specific scripting and management toolset

• SUSE Linux• Suse Cloud – Chef based toolset

• And more…

StackForge

• Tools for installing on your own• A project under OpenStack CI that includes many “pieces” of projects• Puppet based options (e.g. Puppetlabs pieces, other contribuilted modules)• Chef based options (crowbar, etc.)• Other non-incubated projects

• https://github.com/stackforge

Thanks!