using docker for gpu accelerated applications

Felix Abecassis, Systems Software EngineerJonathan Calmels, Systems Software Engineer

USING DOCKER FOR GPU ACCELERATED APPLICATIONS

2

AGENDA

GPU COMPUTINGCUDAEcosystemApplications

NVIDIA DOCKERChallengesOur Solution

DEMOSGPU IsolationMachine LearningRemote Deployment

3

GPU COMPUTINGnvidia.com/object/gpu-accelerated-computing.html

4

HETEROGENEOUS COMPUTING

GPU CPU

+

Optimized forparallel tasks

Optimized for serial tasks

5

// Vector sum in Cvoid vector_add(int n, const float* a, const float* b, float* c){ for (int idx = 0; idx < n; ++idx) c[idx] = a[idx] + b[idx];}

// Vector sum in CUDA__global__ void vector_add(int n, const float* a, const float* b, float* c){ int idx = blockIdx.x * blockDim.x + threadIdx.x; if (idx < n) c[idx] = a[idx] + b[idx];}

CUDA C++ PROGRAMMING

6

ECOSYSTEM

AmgX cuBLAS

LIBRARIES

COMPILER DIRECTIVES

PROGRAMMING LANGUAGES

x86

7

NVIDIA SDKs

8

APPLICATIONS: DEEP LEARNING

INTERNET & CLOUDImage ClassificationSpeech RecognitionLanguage TranslationLanguage ProcessingSentiment AnalysisRecommendation

MEDICINE & BIOLOGYCancer Cell DetectionDiabetic GradingDrug Discovery

MEDIA & ENTERTAINMENTVideo CaptioningVideo SearchReal Time Translation

SECURITY & DEFENSEFace DetectionVideo SurveillanceSatellite Imagery

AUTONOMOUS MACHINESPedestrian DetectionLane TrackingRecognize Traffic Sign

9

GPU-ACCELERATED DEEP LEARNING

TENSORFLOW CNTK TORCH CAFFE

WATSON CHAINER THEANO MATCONVNET

10

NVIDIA DOCKERgithub.com/NVIDIA/nvidia-docker

11

CHALLENGESA Typical Cluster

Ubuntu 14.04Drivers 3524x Maxwell

CentOS 7Drivers 3464x Kepler

Ubuntu 16.04Drivers 3628x Pascal

CUDA 7.5CUDA 7.0cuDNN 3

CUDA 7.5cuDNN 4 CUDA 8.0

Patches

12

PACKAGING DRIVER FILES?

FROM ubuntu:14.04

RUN apt-get update && \apt-get install --no-install-recommends -y gcc make libc-dev

wget

RUN wget http://us.download.nvidia.com/XFree86/Linux-

x86_64/361.42/NVIDIA-Linux-x86_64-361.42.runRUN sh NVIDIA-Linux-x86_64-361.42.run --silent --no-kernel-module

Never install the driver in the Dockerfile, not portable!

13

BRINGING GPU SUPPORT TO DOCKER

14

INTERNALS

Docker Daemon

NVIDIA Docker Plugin

NVIDIA Drivers

NVIDIA Docker

Docker Registry

1

2

3

4

5

6

HTTP + SSH

HTTP + UNIXLDCACHE + NVML

HTTPS

CUDA 8.0CAFFE

CUDA 7.5TENSORFLOW

HTTPS

HTTPSDocker CLI

15

DOCKERHUB IMAGES

CUDA 7.0 runtime

CUDA 7.5 runtime

CUDA 8.0 runtime

cuDNN v4 runtime

CUDA 7.0 devel

cuDNN v4 runtime

CUDA 7.5 devel

cuDNN v5 runtime

CUDA 8.0 devel

NVIDIA/Caffe0.14.2

cuDNN v4 devel

cuDNN v4 devel

NVIDIA/Caffe0.14.5

cuDNN v5 devel

DIGITS3.0 CNTK BVLC/Caffe TensorFlow DIGITS

3.3

Ubuntu14.04

Ubuntu16.04

16

GPU APPLICATIONS WORKFLOW

Research/Develop Test/Package Deploy

Video Transcodin

g

Image Processing

Deep Learning HPC

Visualization

Container Based Applications

GPU-Accelerated Data Center

17

LEARN MOREwww.nvidia.com/dgx1

THANK YOU