PowerPoint Presentation

HPC to OpenStackOur journeyTuesday 16th February 2016

Outline

Key Questions

Provisioning Systems

Configuration Management

Futures

Q&A

How do we??

Install bare metal

Configure networking, and base OS

Install software

Manage the configuration

Make it redundant

Provisioning Systems

Which one to chooseForeman

Cobbler

Oscar

Rocks

xCAT

PCM

Bright

Others...

What is xCAT

eXtreme Cluster/Cloud Administration Toolkit

Management of clusters

Baremetal Provisioning

Scriptable

Large scale management (Lightsout, remote console, distributed shell)

Configures key services based on tables

Why we're using xCAT

xCAT provides bare metal management and provisioning of hardware.

Automatic Discovery

EPL LicencingAdaptable

No licencing cost

Close relationship with developers

Hardware managementLights out management

Remote console/Serial over Lan

Remote inventory, vitals, power

Pros and Cons

ProsFlexible

Open-Source

HW Support

OS Support

ConsNo GUI

No Configuration Management

Very Steep Learning Curve

Only maintained by IBM/Lenovo

How is this relevant to OpenStack

The installation process is very similar

The quantity of nodes are very similar

We still need to provision nodes from bare-metal

XCAT has an Ironic driver :)

Why NOW!!!!

KVM performance

SR-IOV support

VXLAN offloading

Why OpenStack for HPC

Users want flexibility

Users have a choice

Users don't depend on Administrators as much

Not all applications are available on the system

Sequential jobs

Embarrassingly parallel jobs

Performance

MVAPICH2-virt performance is approx 1-4% slower compared to native (source Mellanox booth SC15)

8% slower w/SR-IOV vs Native IB (source SDSC) compared to 28% 2 years ago on KVM

http://mvapich.cse.ohio-state.edu/static/media/talks/slide/dk_mellanox_theatre_sc15_1.pdf

Our Dilemma

OpenStack is so new

Not many people know it

Where do we start

Our Choices

Initially (Havana - Icehouse)CentOS

RDO

Packstack

Middle (Icehouse Juno)CentOS

RDO

Crudini (openstack-config)

Then (Juno - Liberty)CentOS

RDO

SaltStack

Why!!!

CentOSBecause a lot of our customers used it

Compatible with GPFS

RDONatural choice after choosing CentOS

Easy way of getting all packages installed via package manager

SaltStackCustomer introduction to the SW

How Strike 1

Packstack

Great for PoC environment

Test any specific features

Simple to get everything running

How Strike 2

Crudini (openstack-config)

A lot of work required to get things up and running

Everything was scripted

Doesn't work with multiple values for any option

Very difficult to maintain and version control

How Strike 3

SaltStack

Created many salt formulas specific for our requirements

Moved all our configsOpenStack

GPFS

Icinga (Nagios)

Ganglia

3rd-party applications

HPC specific formulas

InfiniBand

Preparing the System

Decide the version of OpenStack

Sync repos by using reposync from the reposepel

RDO

CentOS

Sample yum.conf and repo files belowhttps://gitlab.arif-ali.co.uk/arif/openstack-lab

Preparing the xCAT

Install xCAT RPMs

Configure xCAT tablesnodes and their configurations

networks

switch mappings

Etc

Install nodes with base OS

Use salt to configure all nodes depending on xCAT groups

https://gitlab.arif-ali.co.uk/arif/openstack-lab

Future Development

Update configs to use salt formulas

Contribute to the salt-formula-* projectsAdd any experience from field

Creating spec files for the salt-formulas

Keystone v3 API integrationFederation support

Use openstack-ansible ??

Finally

Use heat to deploy virtual HPC environments

Create private networks

Install custom scheduler and software

Create as many as you want

https://gitlab.arif-ali.co.uk/arif/openstack-lab/tree/master/heat_templates

Links

https://gitlab.arif-ali.co.uk/arif/openstack-lab

http://blog.arif-ali.co.uk/blog/categories/openstack

https://gitlab.arif-ali.co.uk/arif/openstack-lab/wikis/home

Q & A

17/02/16

Click to edit Master title style

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelClick to edit Master text stylesSecond levelThird levelFourth level

Fifth level

17/02/16

17/02/16

17/02/16