USING CONTAINER-NATIVE STORAGE:MYTHS AND BEST PRACTICESHow to Care for Your Storage

José A. RiveraSenior Software Engineer2018.04.10

or

OVERVIEW

● ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

OVERVIEWPlatform-agnostic Storage

● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Scalable (1000+ volumes)

Highly-Available(across availability zones)

Automated (Dynamic Provisioning)

Integrated(installs with / runs on OpenShift)

OPENSHIFT NODE 2 OPENSHIFT NODE 3

OPENSHIFT NODE 1

RHGS Container RHGS Container

OPENSHIFT NODE 4

MASTERAPP Container APP Container

APP CONTAINER

RHGS Container

OVERVIEWContainer-Native Storage

● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

“A request for storage”

Provider: ABCCapacity: 10 GiBFeatures: XYZ

PersistentVolumeClaim

“A provider of storage”

Provider URL: …Credentials: …Options: ...

StorageClass

“Provisioned Storage”

Name: …Size: …AccessMode: ...

PersistentVolume

APPLICATION POD(S)

“sets up”

“submits” “submitted to” “creates”

Storage Backend

“instructs” “provisions”

“mounted by”

OVERVIEWDynamic Provisioning

● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

MYTHS

● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

MYTHSCollocation means increased performance

MYTH:Collocating my application pods with my storage pods will increase storage I/O performance.

● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○

MYTHSCollocation means increased performance

MYTH:Collocating my application pods with my storage pods will increase storage I/O performance.

FACT: Only for read-only use cases, and not a significant amount.

● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○ ○

MYTHSStorage should be isolated

MYTH:Storage nodes should only host CNS pods.

● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○

MYTHSStorage should be isolated

MYTH:Storage nodes should only host CNS pods.

FACT: For the vast majority of cases, storage nodes can be treated like any other app or infrastructure node.

● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○ ○

MYTHSSplit devices between multiple clusters

MYTH:I can isolate different types of devices on the same node in multiple CNS clusters.

● ● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○

MYTHSSplit devices between multiple clusters

MYTH:I can isolate different devices on the same node in multiple CNS clusters.

FACT: A given storage node may only run one RHGS pod, and thus can only be part of a single cluster.

● ● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○ ○

MYTHSIsolate volume types

MYTH:I should isolate different types of storage volumes into their own clusters.

● ● ● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○

MYTHSIsolate volume types

MYTH:I should isolate different types of storage volumes into their own clusters.

FACT: A single CNS cluster can provide a variety of volume types.

● ● ● ● ● ● ● ● ● ● ○ ○ ○ ○ ○ ○

BEST PRACTICES

● ● ● ● ● ● ● ● ● ● ● ○ ○ ○ ○ ○

BEST PRACTICESStorage node specs

Each storage node should meet the following specifications:

● MEMORY: 8 GB of RAM + 30 MB of RAM per active volume○ This counts all volumes in the cluster, even if a given node is not hosting any

data for some of those volumes● CPU: No technical minimum, recommended minimums are:

○ Intel Xeon Nehalem-Ex or equivalent○ Dual socket server○ Quad-core processors

These minimums account for OS needs and other applications running on the same node.

● ● ● ● ● ● ● ● ● ● ● ● ○ ○ ○ ○

BEST PRACTICESCluster size

Cluster size is heavily dependent on anticipated workloads, but there are some recommended minimums and sizing guidelines:

● Technical minimum of 3 nodes using default configuration● Recommended minimum of 4 nodes to increase failure tolerance● Two separate clusters are recommended, one for user applications and one for

infrastructure applications● Recommended maximum of 1000 concurrent volumes per 3-node cluster

○ Max does not scale linearly with additional nodes○ Recommended to add additional clusters to support > 1000 volumes

● ● ● ● ● ● ● ● ● ● ● ● ● ○ ○ ○

BEST PRACTICESIncreasing capacity

CNS provides various ways to increase storage capacity:

● Add more devices to existing nodes● Add more nodes to existing clusters

○ Maximum 128 nodes per cluster● Add more clusters to existing CNS installations

○ Can be immediately used by existing StorageClasses● Deploy additional CNS installations

○ Requires namespace isolation

Rough maximum of 17 Petabytes of total storage capacity using three-way replicated volumes.

● ● ● ● ● ● ● ● ● ● ● ● ● ● ○ ○

QUESTIONS?

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ○

THANKS!

José A. Rivera <jarrpa@redhat.com> @jarrpa

Container-Native Storagehttps://www.redhat.com/en/technologies/storage/use-cases/container-native-storage

OpenShift Advanced Installer - GlusterFS Documentationhttps://docs.openshift.com/container-platform/3.9/install_config/persistent_storage/persistent_storage_glusterfs.html

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●