timothy carros: ubuntu 14.04 lts ubuntu juju (5) hypervisor: qemu openvswitch ceph (rbd) deployment...
TRANSCRIPT
● Provide a dynamic and elastic compute / storage environment for researchers.
● Get rid of “under the desk“ syndrome.
● Help build platforms and capabilities to deliver them in a sustainable manner for research consumption.
Solving the lack of Research
Compute
South African Data-Intensive Reseach Cloud (SADIRC)
● Expanding the ARC by onboarding SKA South Africa and SANSA.
● The main driver behind SADIRC is to provide collaborative access to all South African researchers including those at under-resourced institutions.
● ARC was a prototype and will now form part of SADIRC.
● Openstack
○ IAAS + Research Hardware (GPUs)
○ Research environments can be tailored accordingly.
○ Openstack is just a set of APIs which provides a level of abstraction to infrastructure.
○ Many development projects now build for some type of cloud of which Openstack generally features.
Solution
● Configuration deployment:○ MAAS ( Metal as a Service)○ OS: Ubuntu 14.04 LTS○ Ubuntu Juju (5)○ Hypervisor: QEMU ○ OpenVSwitch ○ Ceph (RBD)
Deployment● Juju Charms
○ Lots of limitations with updating charms.○ Looking to move away - Deploy with
Ansible. ○ 1500 MTU virtual switch issue, L&M Rel○ Federated Authentication.
● ARC Core Services:○ x3 Dell R620○ x3 LXC containers of the following:
■ RabbitMQ, Percona MySQL■ Nova, Keystone, Glance ■ Keystone, Dashboard, etc etc..
○ Juju resolves failed services with HA.
● Written in GOlang.● Build and maintain machine images.● Ships as a single binary Packer is not
a configuration management replacement.
● Multitude of builders - Microsoft Azure, GCP, AWS, Openstack
● IDIA images are baked and then deployed using Terraform rolling updates.
Terraform.io ● Architect your entire infrastructure stack in code.
● Written in GOlang.● Single binary, no dependencies.● Multiple Providers -Microsoft
Azure, GCP, AWS, Openstack.● Scale your environment.
singularity.lbl.gov
● Simple installation.● Container images can only be
built as root. ● Read-only for non-root users. ● Host storage and networking
stack used by Singularity, no passthrough required.
● UID/GID mapping to the container● Storage taken care of for you. ● Native support for GPUs / IB and
X server applications.
singularity.lbl.gov
● Maintenance of singularity containers through the use of makefiles.
IMAGE=jupyter-casa.imgDEF=jupyter-casa.def
.PHONY: bootstrap
all: bootstrap
${IMAGE}:sudo singularity create -s 6000 ${IMAGE}
bootstrap: ${IMAGE}sudo singularity bootstrap ${IMAGE} ${DEF}
upgrade:sudo singularity exec --writable ${IMAGE} apt-get updatesudo singularity exec --writable ${IMAGE} apt-get upgradesudo singularity exec --writable ${IMAGE} conda update --all
clean: sudo rm ${IMAGE}
● Jupyter Kernels are hosted inside Singularity containers.
● A more complex suite of astronomy applications are compiled and built into these singularity containers.
● Containers are stored on the BeeGFS volume and shared across the entire suite of IDIA worker machines.
{ "argv": ["/data/exp_soft/containers/jupytercasa.img" "-m", "casapy", "-f", "{connection_file}"], "mimetype": "text/plain", "display_name": "Juypter-Casa", "language": "casa"}
jupyter/kernels/JupyterCasa/kernel.json
● Shared BeeGFS Posix Storage. ● User Authentication SSH public key stored
in LDAP, ldappasswd for web-based services.
● Astronomy applications packaged in containers, stored on shared file system.
● Jupyter Hub research execution portal.● Terraform python client used to scale
resources. ● CLI to execute container apps.
IDIA Research Environment