ceph and cloud stack apr 2014
DESCRIPTION
Ceph and CloudStack - from CloudStack Collaboration SummitTRANSCRIPT
![Page 1: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/1.jpg)
20140411
![Page 2: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/2.jpg)
Ian Colle Director of Engineering, Inktank [email protected] @ircolle www.linkedin.com/in/ircolle ircolle on oftc/freenode inktank.com | ceph.com
![Page 3: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/3.jpg)
AGENDA
3
![Page 4: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/4.jpg)
CEPH
![Page 5: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/5.jpg)
CEPH UNIFIED STORAGE
FILE SYSTEM
BLOCK STORAGE
OBJECT STORAGE
CloudStack
Geo-Replication
Native API
5
Multi-tenant
S3 & Swift
CloudStack
Linux Kernel
iSCSI
Clones
Snapshots
CIFS/NFS
HDFS
Distributed Metadata
Linux Kernel
POSIX
Copyright © 2014 by Inktank
![Page 6: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/6.jpg)
CEPH OVERVIEW
PHILOSOPHY TODAY
6
2004
2010
2012
HISTORY
2006
Included in Linux kernel
Integrated into CloudStack
Open sourced for the first time
Project starts at UCSC
Failure is normal Self managing Scale out on commodity hardware
Everything runs in software
Copyright © 2014 by Inktank
![Page 7: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/7.jpg)
TRADITIONAL STORAGE VS. CEPH
TRADITIONAL ENTERPRISE STORAGE
7 Copyright © 2014 by Inktank
![Page 8: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/8.jpg)
STRONG & GROWING COMMUNITY
8
8,172
37,946
2,888
11,500
1,418 2,715
2011-Q3 2011-Q4 2012-Q1 2012-Q2 2012-Q3 2012-Q4 2013-Q1 2013-Q2 2013-Q3
IRC chatlines
ML messages
Commits
Copyright © 2014 by Inktank
![Page 9: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/9.jpg)
ARCHITECTURE
9
S3/SWIFT HOST/HYPERVISOR iSCSI CIFS/NFS SDK
INTE
RFAC
ES
STOR
AGE
CLUS
TERS
MONITORS OBJECT STORAGE DAEMONS (OSD)
BLOCK STORAGE FILE SYSTEM OBJECT
STORAGE
Copyright © 2014 by Inktank
![Page 10: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/10.jpg)
CRUSH
10
hash(object name) % num pg
CRUSH(pg, cluster state, rule set)
![Page 11: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/11.jpg)
11
![Page 12: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/12.jpg)
§ CRUSH § Pseudo-random placement algorithm
§ Fast calculation, no lookup § Repeatable, deterministic
§ Statistically uniform distribution § Stable mapping
§ Limited data migration on change § Rule-based configuration
§ Infrastructure topology aware § Adjustable replication § Weighting
12
![Page 13: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/13.jpg)
13
??
![Page 14: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/14.jpg)
14
![Page 15: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/15.jpg)
15
![Page 16: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/16.jpg)
16
??
![Page 17: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/17.jpg)
17
![Page 18: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/18.jpg)
18
![Page 19: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/19.jpg)
19
![Page 20: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/20.jpg)
20
![Page 21: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/21.jpg)
HOW DO YOU SPIN UP
HUNDREDS OF VMs INSTANTLY
AND EFFICIENTLY?
21
![Page 22: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/22.jpg)
22
instant copy
= 144
![Page 23: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/23.jpg)
23
write
write
write
= 148
write
![Page 24: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/24.jpg)
24
read
read
read
= 148
![Page 25: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/25.jpg)
CEPH AND CLOUDSTACK
![Page 26: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/26.jpg)
§ Wido den Hollander [email protected] § If you don’t know him, you should
§ Developed § rados-java § libvirt support for RBD § CloudStack integration
THIS IS THE HOUSE THAT WIDO BUILT
![Page 27: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/27.jpg)
ARCHITECTURAL COMPONENTS
27
APP HOST/VM CLIENT
Copyright © 2014 by Inktank
![Page 28: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/28.jpg)
CEPH WITH CLOUDSTACK
28 Copyright © 2014 by Inktank
![Page 29: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/29.jpg)
§ Ceph (Firefly in rc) § http://www.ceph.com/download/
§ libvirt (1.2.3 released on 01APR2014) § http://libvirt.org/sources/
§ QEMU (2.0 in rc, or 1.7.1) § http://wiki.qemu.org/Download
§ Cloudstack (4.2 or newer) § http://cloudstack.apache.org/downloads.html
WHAT DO I NEED?
![Page 30: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/30.jpg)
§ Set up a ceph cluster § http://ceph.com/docs/master/start/
§ Install/configure QEMU § http://ceph.com/docs/master/rbd/qemu-rbd/
§ Install/configure libvirt § http://ceph.com/docs/master/rbd/libvirt/
§ Create your Cloudstack pool § ceph osd pool create cloudstack
§ Add Primary Storage § For Protocol select RBD
WHAT DO I DO?
![Page 31: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/31.jpg)
WHAT’S NEXT FOR CEPH?
![Page 32: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/32.jpg)
CEPH ROADMAP
32
Firefly Giant H-Release
Copyright © 2014 by Inktank
![Page 33: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/33.jpg)
CACHE TIERING - WRITEBACK
Copyright © 2014 by Inktank
5 PB HDD OBJECT STORAGE
500TB writeback cache
![Page 34: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/34.jpg)
CACHE TIERING - READONLY
Copyright © 2014 by Inktank
5 PB HDD OBJECT STORAGE
200TB redonly cache
150TB readonly cache
150TB readonly cache
![Page 35: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/35.jpg)
35
Costs you 30MB of storage
![Page 36: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/36.jpg)
36
Costs you ~14MB of storage
![Page 37: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/37.jpg)
NEXT STEPS
![Page 38: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/38.jpg)
NEXT STEPS WHAT NOW?
• Read about the latest version of Ceph: http://ceph.com/docs
• Deploy a test cluster using ceph-
deploy: http://ceph.com/qsg
§ Most discussion happens on the mailing lists ceph-devel and ceph-users. Join or view archives at http://ceph.com/list
§ IRC is a great place to get help (or help others!) #ceph and #ceph-devel. Details and logs at http://ceph.com/irc
38
• Deploy a test cluster on the AWS free-tier using Juju: http://ceph.com/juju
• Ansible playbooks for Ceph: https://github.com/ceph/ceph-ansible
§ Download the code: http://www.github.com/ceph
§ The tracker manages bugs and feature requests. Register and start looking around at http://tracker.ceph.com
§ Doc updates and suggestions are always welcome. Learn how to contribute docs at http://ceph.com/docwriting
![Page 39: Ceph and cloud stack apr 2014](https://reader034.vdocuments.net/reader034/viewer/2022052301/5550c417b4c90504628b55fa/html5/thumbnails/39.jpg)
Ian R. Colle Director of Engineering
[email protected] @ircolle
www.linkedin.com/in/ircolle ircolle on oftc/freenode