![Page 1: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/1.jpg)
Michael Bauer - HPC Container Workshop ISC19
Singularity, SLURM, and Kubernetes - 5 min20 June 2019
![Page 2: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/2.jpg)
Running Singularity with K8s
Singularity & Kubernetes
● Singularity-CRI now in v1.0.0-alpha.3 release
○ Please try and provide feedback!
● Run all workloads (K8s microservices, HPC, EPC, etc…) with
one unifying runtime
● Native support for GPUs, Infiniband/RDMA (coming soon),
other HPC-oriented hardware via Container Device Plugin
![Page 3: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/3.jpg)
Kubernetes - scheduling to multiple HPC clusters - using Singularity containers to distribute the workloads.
All as unprivileged user.
Kubernetes & SLURM
Multi-Cluster Scheduling
![Page 4: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/4.jpg)
● Submit SLURM job request to Kubernetes server using new
job kind “SlurmJob”
● Custom K8s Resource Daemon lives on each SLURM master
node, lets K8s understand what resources exist
● Use default or custom K8s scheduling algorithm to dispatch
jobs to different clusters
Singularity, SLURM, K8s
Multi-Cluster Scheduling to SLURM via K8s
![Page 5: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/5.jpg)
Singularity, SLURM, K8s
![Page 6: 20 June 2019 - QNIBqnib.org/data/hpcw19/4_SCHED_5_Singularity-Slurm.pdf · UBUNTU CODENAME-=bionic Singularity "—THE UNIVERSIIY OF IOWA UNIVERSITY OF MICHIGAN amazon web services](https://reader036.vdocuments.net/reader036/viewer/2022090605/605ab28494c0aa01545cb079/html5/thumbnails/6.jpg)
K8s SlurmJob API
apiVersion: slurm.sylabs.io/v1alpha1
kind: SlurmJob
metadata:
name: cow
spec:
batch: |
#!/bin/sh
##SBATCH --nodes=1 --cpus-per-task=1
srun singularity pull library://sylabsed/examples/lolcow
srun singularity run lolcow_latest.sif
srun rm lolcow_latest.sif
nodeSelector:
containers: singularity