openattic technology overview - ceph management
TRANSCRIPT
![Page 1: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/1.jpg)
openATTIC Technology Overview - Ceph ManagementLenz Grimmer <[email protected]>Vault Conference 2016, Raleigh (NC), USA
![Page 2: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/2.jpg)
2
openATTIC – History / Motivation / Background
─ Development started ~5 years ago
─ Develop a free alternative to commercial / traditional
storage management systems
─ Based on Linux / OSS tools
─ Development sponsored by it-novum
─ „Traditional Storage“ and Scale-out
![Page 3: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/3.jpg)
3
─ Modern Web UI─ Software-Defined Storage:
RESTful API─ Unified Storage
─ NAS (NFS, CIFS)─ SAN (iSCSI, FC)
─ LVM, XFS, ZFS, Btrfs, ext3/4
─ Ceph Support: CRUSH map, Pool/RBD management (WIP)
─ Mirroring (DRBD®)─ Clustering
(Pacemaker/Corosync)─ Monitoring (Nagios) built-in─ Open Source & commercial
Support
openATTIC – Open Source Storage Management
![Page 4: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/4.jpg)
4
openATTIC Components – Backend
─ Django (Python)─ Django REST Framework (RESTful API)─ Various Linux tools for storage management, e.g. LVM, LIO,
filesystem utilities, DRBD, etc.─ Nagios/Icinga & PNP4Nagios (Monitoring and Graphing)
![Page 5: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/5.jpg)
5
openATTIC Components – Frontend
─ AngularJS (JS framework)─ Bootstrap (HTML, CSS, and JS framework)─ Uses REST API exclusively─ Bower (JS package manager, NodeJS)─ Grunt (Automation, NodeJS)─ Protractor / Jasmine (UI Test framework)
![Page 6: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/6.jpg)
6
Django REST Framework
Backend/Models(Django)
Client (GUI)
oA-systemd
REST API
D-Bus
Node 1
OS Commands
Django REST Framework
Backend/Models(Django)
Custom App
oA-systemd
REST API
D-Bus
Node 2
OS Commands
MultinodeInterconnect
openATTIC – Architecture
DB(PostgreSQL)
![Page 7: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/7.jpg)
7
openATTIC – Ceph Management Challenges
─ How to find the best approach for managing Ceph─ An ISV‘s view
─ Distributed systems vs. storing configuration data in a local
database─ Allow external modifications / administration of the Cluster─ How to maintain consistency of a local configuration DB?
![Page 8: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/8.jpg)
8
openATTIC – Choosing the right Ceph API
─ Option 1: Calling CLI tools and parsing output (JSON-
format)
─ Option 2: Use existing APIs and libs, e.g.─ librados (Python bindings)─ Ceph-REST-API─ Calamari Server
─ API documentation
─ Redundancy / Scalability / Dependencies
─ Which is the best choice?
![Page 9: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/9.jpg)
9
openATTIC – Other Ceph Management Tools
─ How are others doing this?
─ Calamari Server / Romana
─ Intel VSM
─ Inkscope / Kraken / Ceph-Dash
![Page 10: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/10.jpg)
10
openATTIC - Ceph Management Implementation
─ Based on librados Python bindings─ Low overhead
─ Implemented “NoDB” Django Models─ All benefits of Django models (e.g. relations, methods, pagination)─ REST API viewsets─ Query Ceph for the model state instead of local DB
─ Use Salt for remote execution on nodes
![Page 11: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/11.jpg)
11
Django REST Framework
Django Models
Client (GUI)
REST API
openATTIC – “nodb” Django models
DB(PostgreSQL)
nodb
Python librados django.db.models
![Page 12: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/12.jpg)
12
openATTIC - Current Development Status
─ “Nodb” Architecture / Framework in place─ Pool Management REST API
─ List Pools, view Pool details─ Create/Delete/Update (WIP)
─ Pool Management Web UI (table view)
![Page 13: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/13.jpg)
13
openATTIC – Future/ongoing Development Work
─ Ceph Cluster Status Dashboard─ Extend Pool Management (Wizard)─ OSD Monitoring/Management─ RBD Management─ CephFS Management─ RGW Management (users, buckets keys)
![Page 14: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/14.jpg)
14
openATTIC – Dashboard
![Page 15: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/15.jpg)
15
openATTIC – Ceph Pool List
![Page 16: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/16.jpg)
16
openATTIC – Ceph CRUSH map editing
![Page 17: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/17.jpg)
17
openATTIC – Wizards
![Page 18: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/18.jpg)
18
openATTIC - Volume View
![Page 19: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/19.jpg)
19
openATTIC - Volume View (Performance)
![Page 20: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/20.jpg)
20
openATTIC – API Recorder
![Page 21: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/21.jpg)
21
www.openattic.org demo.openattic.org blog.openattic.org docs.openattic.org
bitbucket.org/openattic tracker.openattic.org @openATTIC openATTIC
openATTIC – Resources
![Page 22: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/22.jpg)
22
Questions / Discussion
![Page 23: openATTIC Technology Overview - Ceph Management](https://reader031.vdocuments.net/reader031/viewer/2022030207/58a9307e1a28ab3f778b5291/html5/thumbnails/23.jpg)
23
Thank you!