the future of cloud software defined storage with ceph: andrew hatfield, red hat
TRANSCRIPT
The Future of Cloud Software Defined
Storage withCeph
2
Andrew Hatfield
Practice Lead – Cloud Storage and Big Data (ANZ)
Red Hat
@andrewhatfield
/usr/bin/whoami
3
Overview
● What is Ceph
● Integration with OpenStack
– Stuff you already know is cool
● New stuff coming
– Even more cool stuff!
4
What is Ceph
5
PROPRIETARY HARDWARE
PROPRIETARY SOFTWARE
SUPPORT & MAINTENANCE
COMPUTER DISK
COMPUTER DISK
COMPUTER DISK
STANDARDHARDWARE
OPEN SOURCE SOFTWARE
ENTERPRISEPRODUCTS & SERVICES
COMPUTER DISK
COMPUTER DISK
COMPUTER DISK
A Storage Revolution
6
CEPH COMPONENTS
RGWweb services gateway
for object storage, compatible with S3 and
Swift
LIBRADOSclient library allowing apps to access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOSsoftware-based, reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDreliable, fully-distributed block device with cloud
platform integration
CEPHFSdistributed file system with POSIX semantics
and scale-out metadata management
APP HOST/VM CLIENT
RADOS CLUSTER
7
APPLICATION
M M
M M
M
RADOS CLUSTER
OBJECT STORAGE DAEMONS
8
FS
DISK
OSD
DISK
OSD
FS
DISK
OSD
FS
DISK
OSD
FS
M
M
M
RADOS COMPONENTS
9
OSDs: 10s to 10,000s in a cluster One per disk (or one per SSD, RAID group…) Serve stored objects to clients Intelligently peer for replication & recovery
Monitors: Maintain cluster membership and state Provide consensus for distributed decision-
making Small, odd number These do not serve stored objects to clients
M
CRUSH: DYNAMIC DATA PLACEMENT
10
CRUSH: Pseudo-random placement algorithm
Fast calculation, no lookup Repeatable, deterministic
Statistically uniform distribution Stable mapping
Limited data migration on change Rule-based configuration
Infrastructure topology aware Adjustable replication Weighting
DATA IS ORGANIZED INTO POOLS
11
CLUSTER
OBJECTS
10
01
01
10
10
01 11
01
1001
0110 10 01
11
01
POOLS(CONTAINING PGs)
10
01
11
01
10
01
01
10
01
10
10
01
11
01
10
01
10 01 10 11
01
11
01
10
10
01
01
01
10
10
01
01
POOLA
POOLB
POOL C
POOLD
OBJECTS
OBJECTS
OBJECTS
12
Integration with OpenStack
THE RADOS GATEWAY
13
M M
M
RADOS CLUSTER
RADOSGWLIBRADOS
socket
RADOSGWLIBRADOS
APPLICATION APPLICATION
REST
RADOSGW MAKES RADOS WEBBY
14
RADOSGW: REST-based object storage proxy Uses RADOS to store objects API supports buckets, accounts Usage accounting for billing Compatible with S3 and Swift applications
RBD STORES VIRTUAL DISKS
15
RADOS BLOCK DEVICE: Storage of disk images in RADOS Decouples VMs from host Images are striped across the cluster (pool) Snapshots Copy-on-write clones Support in:
Mainline Linux Kernel (2.6.39+) Qemu/KVM, native Xen coming soon OpenStack, CloudStack, Nebula, Proxmox
KERNEL MODULE FOR MAX FLEXIBLE!
16
M M
RADOS CLUSTER
LINUX HOSTKRBD
STORING VIRTUAL DISKS
17
M M
RADOS CLUSTER
HYPERVISORLIBRBD
VM
CEPH AND OPENSTACK
18
RADOSGWLIBRADOS
M M
RADOS CLUSTER
OPENSTACKKEYSTONE CINDER GLANCE NOVASWIFT
LIBRBD LIBRBD
HYPER- VISOR
LIBRBD
19
What is New()?
20
Recent news
● Ceph continues to be the #1 storage choice for Open Stack (April User Survey)
● Ceph is now an openstack.org Reference Architecture (joining LVM & NFS)
● Ceph at Percona Live for MySQL Database-as-a-Service
● CephFS is now stable and ready for testing (Jewel release)
● ceph-ansible deployment in 2 commands
21
● OpenStack Manila + CephFS for File-as-a-Service
● BlueStore – Direct to block device, no filesystem
● Active / Active Multi-master RGW for global object stores
● NFS Gateway for RGW (v4 only) for legacy access
● Active Directory support for RGW
● AWS4 Support
● Swift – Keystone v3, Object expiration, tenant namespaces, static large objects
● RBD mirroring
● Client IO prioritised over scrubbing
● Containerised Ceph
Work in Progress
22
THANK YOU!
Andrew HatfieldPractice Lead - Cloud Storage and Big Data
@andrewhatfield