Bringing Private Cloud to Australia

OpenStack on VMware

OpenStack Summit 2013

Introduction

• Aptira• Leading OpenStack provider in Australia and APAC• Private and Hybrid IaaS Cloud Solutions• Technology consultancy for large providers with huge scale and load requirements• Customers in Finance, Retail, Utilities, Government, Rich Media Producers• Founder/prime motivator of the OpenStack community in Australia and India

• Sina Sadeghi• Director, Cloud Operations• I don't only write the puppet module, also need to determine which technologies are a

commercially viable solution for customers and where they fit in our strategy• Aptira employee since August 2012• Previously: sysadmin/ops for major Australian research and commercial institutions • Part of team that deployed the NeCTAR Research Cloud on OpenStack – currently 30,000

cores and 3,000 users.

Perception

• Internal use-case• existing successful and growing business running on vSphere

1. Buy new hardware, lease new racks2. Sina deploys greenfields OpenStack3. ???4. Profit!

• External use-case• Build it and they will come

Reality: Internal use-case

• vSphere had features we relied on for our bread and butter (HA, DRS, secure live migration, power management, etc)

• OpenStack relatively immature at the time• Only a few “cattle” customers, mostly “puppies”

customers• Deploying OpenStack would mean duplication of all

existing features in parallel• We love ghettoVCB

Reality: External use-case

• Australia is the most virtualised country per capita in the world

• vSphere approaches 80% of commercial virtualisation solutions

• Huge investments in training, documentation, change control, DR, everything based on vSphere

Does it do vSphere HA?

How does this compare to vSphere?

How much does it cost?

Other

Reality

• OpenStack “Vanilla” only meets requirements for• Greenfields• Service Providers• Public Cloud (or similar)

• Want to use OpenStack?• CapEx for new hardware and datacenter footprint• Completely re-operationalise around OpenStack• Can you afford developer(s) to drive source in the direction you

require?• End up with two silos

Pride/Prejudice

• I am an Open Source geek• So is ~99% of the OpenStack “DevOps” community• VMware purchased Nicira and joined OpenStack

Foundation• Is this VMware versus OpenStack?• Watched VMware CTO promise their support for

OpenStack

OpenStack + VMware

• Hey, actually this is a really good idea!• It's not VMware versus OpenStack (but maybe it is

vCloud versus OpenStack)• Existing driver contributed by Sean Chen • Didn't really seem to work at all• nova-compute/nova-network model doesn't really fit• Let's work with VMware to get this usable!

OpenStack + VMware

• Started working with OpenStack@VMware team (special thanks Dan Wendlandt, Shawn Hartsock)

• Also working closely with Nicira which became VMware Networking and Security Business Unit (special thanks Sumner Burkart, Jacob Cherkas)

• Major focus: Show customers and potential customers that they can expose their existing vSphere clusters as scalable IaaS clouds on open standards without significant CapEx or silo effect.

• It can be done!• Expose existing infrastructure as new technology is a massive resource efficiency• As it should be: ESXi or vCenter “just another hypervisor”• Bring Private Cloud to Australia!

We have come a long way

• Since Dan Wendlandt live-patched our Grizzly code at the Havana summit

• The code works (very well)!• Real documentation!• nova core dev at AOSUG Sydney noted the contributions

as impressive• https://bugs.launchpad.net/nova/+bugs?field.tag=vmware

• http://docs.openstack.org/trunk/config-reference/content/vmware.html

• We are excited to start change control for Havana!

Today - Overview

Today – nova.conf

[DEFAULT]

compute_driver=vmwareapi.VMwareVCDriver

[vmware]

host_ip=<vCenter host IP>

host_username=<vCenter username>

host_password=<vCenter password>

cluster_name=<vCenter cluster name>

datastore_regex=<optional datastore regex>

wsdl_location=http://127.0.0.1:8080/vmware/SDK/wsdl/vim25/vimService.wsdl

Today - Images

1. Download fresh Precise cloud image from: http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img2. Convert to VMDK using "qemu-img convert" (locally on a linux machine)3. Convert using vmware CLI:

vmkfstools.pl ­­server=IPADDRESSOFVMHOSTorHOSTNAME ­i /vmfs/volumes/SOURCEPATH/SOURCEFILENAME.vmdk /vmfs/volumes/DESTINATIONPATH/DESTINATIONFILENAME.vmdk

4. Upload the image to glance:glance image­create ­­name “IMAGENAME” ­­disk­format vmdk ­­container­format bare ­­public ­­property “vmware_adaptertype=lsiLogic” ­­property “vmware_ostype=ubuntuGuest” ­­property “vmware_disktype=eagerZeroedThick”

Today - Images$ glance image­show 5bf97072­d4bf­4030­a685­b36a543de6d9+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+| Property                      | Value                                |+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+| Property 'vmware_adaptertype' | lsiLogicsas                          || Property 'vmware_ostype'      | windows7Server64Guest                || checksum                      | ee6b6f2c59471243e8f2579d74302c13     || container_format              | ovf                                  || created_at                    | 2013­10­11T15:31:17                  || deleted                       | False                                || disk_format                   | vmdk                                 || id                            | 5bf97072­d4bf­4030­a685­b36a543de6d9 || is_public                     | True                                 || min_disk                      | 0                                    || min_ram                       | 0                                    || name                          | WS2012SQL                            || owner                         | bde4b0c3645c49f9a0a2788c6685e40c     || protected                     | False                                || size                          | 16106127360                          || status                        | active                               || updated_at                    | 2013­10­11T23:35:49                  |+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+

Today - Images$ glance image­show d4a13a34­4649­4ec6­8fe9­247e5d3b2ea4+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+| Property                      | Value                                |+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+| Property 'vmware_adaptertype' | lsiLogic                             || Property 'vmware_disktype'    | eagerZeroedThick                     || Property 'vmware_ostype'      | ubuntuGuest                          || checksum                      | e56eed8bd55b3da101d5fbd6ae066e3d     || container_format              | ovf                                  || created_at                    | 2013­07­29T06:02:23                  || deleted                       | False                                || disk_format                   | vmdk                                 || id                            | d4a13a34­4649­4ec6­8fe9­247e5d3b2ea4 || is_public                     | True                                 || min_disk                      | 0                                    || min_ram                       | 0                                    || name                          | Ubuntu 12.04 VMDK                    || owner                         | bde4b0c3645c49f9a0a2788c6685e40c     || protected                     | False                                || size                          | 2147483648                           || status                        | active                               || updated_at                    | 2013­07­29T06:02:47                  |+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+

What works

• We are using Ubuntu 12.04 Precise (with Ubuntu Cloud Archive) virtual machines for OpenStack Grizzly infrastructure (keystone, glance, cinder, horizon)

• We are using https://github.com/openstack-vmwareapi-team/nova.git for nova (all services) – Grizzly

• Keen to go back to using UCA for everything• We are using Nicira provided packages for Neutron and

physical commodity servers for Nicira• All important features work correctly!

What doesn't work

• Often a disconnect between reporting a bug and having that bug implemented in “stable” and then included in UCA

• nova console­log• Image interactions between glance/nova-compute/vmware datastores

leave something to be desired (not sure if technical fix is easy)• Snapshot semantics• Virtual machines don't launch in a tenant associated folder• Existing machines need an on-boarding process• Many minor issues fixed as of Havana!• https://bugs.launchpad.net/nova/+bugs?field.tag=vmware

Tenant-Side

Operator-Side

Benefits of OpenStack API & Ecosystem

Choice of best-in-class virtualization & management technologies

Horizon( Web Portal )

vSphere & vCenter

CLI Tools & Scripts(DevOps Automation)

vCACApplication Director

Nova(Compute)

Neutron(Network)

Cinder(Block Storage)

Glance(Image Store)

NSX vSANvCenter

(Image Catalog)

Cloud Operator Tools(vCenter, vCOPs, Log Insight etc.)

Third Party Operator tools(Puppet/Chef, scripts, nagios...)

OpenStack or 3rd Party ComponentVMware Component

Today: Internal use-case

• Deployed and in production use• Peace of mind for existing technical staff• Offering unique solutions to our customers based on

OpenStack• Big Data• Digital Media Platforms

• Easy to offer PaaS on top (HEAT, scalr, Docker, OpenShift, CloudFoundry) for web hosting

• Exploring multi-hypervisor multi-region configurations

Today: External use-case

• Offer Private Cloud to potential customers with no infrastructure CapEx required

• Australia wants to be able to provision their virtual machines quicker

• Australia wants to be able to provision their virtual networks quicker

• Australia wants to be able to offer utility model to internal customers

• Give Australia a platform to build “cattle” applications on

Fin

• Sina Sadeghi• Aptira: http://www.aptira.com• Twitter: @SinaTweet, @Aptira, @OpenStackAU• https://launchpad.net/~sina-sa• www.meetup.com/Australian-OpenStack-Users-Group