blackstor - world's fastest & most reliable cloud native software defined storage
Post on 18-Feb-2017
337 Views
Preview:
TRANSCRIPT
BlackStor(Based on DRBD)
World's fastest & most reliable Cloud Native Software Defined Storage
How it came into the world, what’s available now and where it’s headed
1
Copyright © 2016 PROZETA & LINBIT
We were simply unhappy with SDS solutions that are available on the market these days… including Ceph.
22
Copyright © 2016 PROZETA & LINBIT
⬢ Since 1991⬢ Group annual turnover: 20M EUR (2015)
⬢ Two divisions➥ Hardware: Value Added Distributor & Reseller➥ Infrastructure services:
IaaS, SaaS, PaaSCE data center infrastructureIn-house development
About PROZETA
5
Copyright © 2016 PROZETA & LINBIT
About LINBIT
7
•Open Source DRBD supported by proprietary LINBIT products / services
•Hundreds of thousands of DRBD downloads
•OpenStack comes with the DRBD Cinder driver
•100% founder owned
•Offices in Europe and US
Best performing SDS block device using common off the shelf hardware
Partners
References
Only replication technology excelling at both synchronous short distance and asynchronous long distance
Copyright © 2016 PROZETA & LINBIT 8
LINBIT’s team is made up of International Open Source experts and has successfully expanded
globally with offices in Europe and the US and partnerships across 5 continents
• LINBIT founded 2001
• European HQ; Vienna (Austria). US HQ; Portland
• Founders are High Availability pioneers
• International OSS, kernel and HA lecturers
• Most recognized Linux HA team worldwide
Incorporation of LINBIT HA
2001
2000 2003
2010
2008 2015
Release of DRBDOpen Source
Lars Ellenberg joins team as Head of Programming
2007
Release of DRBD Proxy
US Office
DRBD inLinux Kernel
DRBD in thousands of clusters worldwide
2016+
DRBD9release
9
DRBD in OpenStack
PROXY
About LINBIT
Copyright © 2016 PROZETA & LINBIT
PROZETA & LINBIT together
9
PROZETAcloud experts
LINBITstorage & HA gurus
COMBINED KNOW-HOW
BEST POSSIBLE STARTING POINT FOR A NEW SDS
Copyright © 2016 PROZETA & LINBIT
SDS Requirements
We wanted SDS on the level of Enterprise storage BUT based on a cost-effective open-source technology we understand.
11
⬢ High Stability⬢ High Reliability⬢ High and Predictable Performance⬢ Scaling options - both Horizontal & Vertical⬢ Modest resource use⬢ Simple to manage (plus repair in case of disaster)⬢ Possible to get info for deep monitoring & metrics⬢ Extensible: features like Compression, Deduplication, etc.
Copyright © 2016 PROZETA & LINBIT
What Are The Options?
13
⬢ IaaS = Volumes / Block Storage
⬢ Servers + Disks + Software
⬢ Local storage
➥ not scalable
⬢ iSCSI
➥ software defined &
redundant: hard to find
⬢ GlusterFS, LizardFS, …
➥ too complex to operate...
Goal:
“Next year I’d like to try out a different storage software solution…
...and I can!”
Copyright © 2016 PROZETA & LINBIT
Situation on the Storage market
14
Open-source:
⬢ Often re-inventing wheel⬢ Too old solutions⬢ Too complex solutions⬢ Over-engineered⬢ Often unstable
Proprietary:
⬢ Often very expensive⬢ Closed source⬢ Not future proof (Software-defined)⬢ What if something goes wrong?
Copyright © 2016 PROZETA & LINBIT
Look at Ceph...
15
● No compression
● No deduplication
● Ineffective thin provisioning
● Extremely CPU hungry
● Bad write performance / high latency - SLOOOOOOOOOW…
High TCO due to the need for extra hardware, unpredictable performance
Perfect integration with OpenStack
VERY stable BUT very difficult to fix potential issues in a timely manner
15
Copyright © 2016 PROZETA & LINBIT 19
Definitely requires a solid underlying base technology …
Something new, right?
19
Copyright © 2016 PROZETA & LINBIT
What is DRBD?
22
⬢ 15+ years on the market⬢ Developed by a stable company with great reputation⬢ More than 250.000 production installations worldwide⬢ Kernel modules (DRBD core) are standard part of Linux kernel⬢ Super-reliable and super-fast!
Copyright © 2016 PROZETA & LINBIT
Why is it so freakin' fast?
24
⬢ Native Linux block device - not a block device emulation on top of an object store
⬢ Highly optimized meta data layout
⬢ The SDS solution leverages on proven technology
➥ LVM or ZFS’ zVols
⬢ Very well suited for hyperconverged compute and storage
Copyright © 2016 PROZETA & LINBIT
DRBD - Key Features
25
⬢ automatic resync after node or connectivity failure➥ direction, amount, no full resync needed
⬢ performs under a Linux kernel implementation➥ 160k IOPs measured (on SSDs, of course)
⬢ multiple volumes per resource (replication group)➥ write order fidelity within resource
⬢ comes with Pacemaker integration⬢ synchronous and async replication (LAN and WAN)⬢ In Linux upstream since 2.6.33 (released 2010)
Copyright © 2016 PROZETA & LINBIT
DRBD - New Features in 9.x
26
⬢ Up to 32 nodes per resource
➥ Fixes the drawbacks of stacking
⬢ Auto promote
⬢ Transport abstraction (TCP, SCTP, RDMA)
⬢ DRBD Manage
Copyright © 2016 PROZETA & LINBIT
DRBD Manage
27
⬢ Provisioning solution for DRBD⬢ Integrated with OpenStack, OpenNebula, Docker, Proxmox, …⬢ Volumes⬢ Snapshots⬢ Replica count⬢ Unlimited number of nodes⬢ Distributed
DRBD Kernel driver
drbdsetup
drbdadm
drbdmanage daemon
device mapper
dmsetup
LVM tools
drbd cinder driver Drbdmanage CLI tool
Copyright © 2016 PROZETA & LINBIT
First implementation
30
⬢ DRBD Kernel modules
+ DRBD Utils
+ DRBD Manage
⬢ DRBD Cinder driver⬢ Monitoring & Metrics⬢ Backup
Copyright © 2016 PROZETA & LINBIT
Stability & Reliability
We have tried to totally break it a million times...
with NO SUCCESS!
Just what you would expect in a production environment
31
Copyright © 2016 PROZETA & LINBIT
BlackStor
34
JSON HTTP CLI WEB UI
API
CONTROL ENGINE MONITORING ENGINE
MESSAGE BROKER
AGENT
DRBD
AGENT
DRBD
AGENT
DRBD
AGENT
DRBD
BI-DIRECTIONAL INTEGRATIONS
TASK MANAGEMENT
SCALING
DISASTER RECOVERY
STATE & PERFORMANCE
METRICS
ANALYTICS
Copyright © 2016 PROZETA & LINBIT
Policy-driven storage
36
⬢ Use simple policies to control:➥ Replication strategy➥ Data placement➥ Balance speed & data protection (RPO)➥ QoS & SLA➥ Backup➥ … and various other knobs
⬢ Assign policies to / by➥ Storage objects➥ Consumer objects
(OpenStack tenant, Cinder volume metadata, ...)
Copyright © 2016 PROZETA & LINBIT
Multi-tenant & Multi-cloud
37
⬢ Use the same storage pool for➥ Multiple OpenStack tenants➥ Multiple OpenStack clusters➥ OpenStack & Docker➥ …..
⬢ We have QoS, so no issues with multi-tenant environment
⬢ Linked authentication➥ Use Keystone for storage authentication➥ Allows OpenStack users to manage resources (if allowed)
Copyright © 2016 PROZETA & LINBIT
Scaling
38
⬢ Horizontally: Scale-out➥ Add nodes to grow cluster➥ Hyper-converged nodes (more nodes, lower capacity per node)
⬢ Vertically: Scale-up➥ Low CPU usage➥ High density storage (high capacity, high iops)➥ 76 TB SSD & 1-2M IOPs in one node? Why not?
⬢ Single volume striped across multiple nodes w/ read balancing➥ Extra boost in performance
Copyright © 2016 PROZETA & LINBIT
QoS
39
⬢ Limit IOPs per➥ Volume➥ Volume group➥ Consumer object (OpenStack tenant, …)➥ Consumer (OpenStack cluster for multi-cluster environment)
⬢ Set priority➥ Set priority per policy
⬢ Alpha version - many limitations but usable⬢ Bandwidth limitation not yet implemented
Copyright © 2016 PROZETA & LINBIT
Backup & “Recycle Bin”
40
⬢ Ability to roll-back without any actual recovery process➥ Storage snapshots (volume roll-back)➥ Deferred delete (volume undelete)
⬢ Backup to another host
➥ Running in background➥ Simple policies➥ No performance degradation➥ Encrypted over the wire
⬢ Recover anything➥ Single volume➥ Full storage
Copyright © 2016 PROZETA & LINBIT
Fully Cloud Aware Storage
41
Everyone's talking about it, but there is none as of now! What it means?
● Bi-directional interfaces (OpenStack, Docker, etc…)● BlackStor keeps all informations about your volumes & VMs inside● Simply define QoS/SLA for OpenStack tenant or VM ….
…. and see if SLA in not breached in the Web UI
Copyright © 2016 PROZETA & LINBIT
Deep insight
42
⬢ Cloud Native - Data + all metadata in one place
⬢ CLI, Web UI⬢ Real-time status
⬢ Metrics➥ IOPs, bandwidth➥ Latency distribution➥ Storage health
(software, hardware)➥ … and many more
Copyright © 2016 PROZETA & LINBIT
BlackStor - Roadmap
43
⬢ Short-term➥ First BlackStor release (incl. new Cinder driver)➥ Multi-tenant, Multi-pool, ACLs, Policy driven➥ Web-based interface, single API, CLI tools➥ Backup, Scale-up & Scale-out➥ Simple installation (DEB/RPM packages, Puppet/Salt recipes)
⬢ Mid/Longer-term➥ Advanced features (QoS)➥ Encryption (both over the wire & data at rest)➥ New drivers (Nova, Docker, Kubernetes, Windows ?)➥ Backup policies
Copyright © 2016 PROZETA & LINBIT
Performance - DRBD by itself
45
● HW○ 2x IBM 8247-22L○ Power 8 2 sockets○ 128 GB RAM○ Mellanox 100GBps InfiniBand○ HGST Ultrastart NVMe SSDs
● Ubuntu Xenial on bare metal● DRBD 9.0.1 & RDMA Transport 2.0● fio 2.2.10● Random IO
Copyright © 2016 PROZETA & LINBIT
Performance - DRBD itselfDRBD with read balancing
48
Read baseline
Copyright © 2016 PROZETA & LINBIT
DRBD vs CEPH
49
⬢ 5-node clusters➥ 2 Class-D SSDs each➥ 2x E5-2650-L CPUs each➥ 128GB RAM each➥ 10GbE each
⬢ Attached VM - 8-core CPU,16GB RAM
⬢ Ubuntu 14.04.5, Kernel 3.16⬢ DRBD9 vs Ceph 10.2.3⬢ fio 2.14
Copyright © 2016 PROZETA & LINBIT 58
...but we went for it anyway.
Why? Because...
➥ nothing reliable out there
➥ great performance & production ready
➥ sweet spot between open source CEPH &
expensive Enterprise storage58
Copyright © 2016 PROZETA & LINBIT 59
What do you get with this solution?⬢ Production ready⬢ Stable & Reliable⬢ Great & Predictable Performance⬢ Simple to implement & manage⬢ Includes deep metrics information⬢ With additional support options
...and last but not least….
⬢ Fantastic TCO - it doesn't cost the Earth!
Copyright © 2016 PROZETA & LINBIT
Thank you!
PRO-ZETA a.s.
Prague, Czech Republic
prozeta@prozeta.eu
www.prozeta.eu
www.tier5.cloud
61
LINBIT HA Solutions GmbH
Vienna, Austria
sales@linbit.com
www.linbit.com
61
Copyright©2016PROZETA&LINBITThankyou!PRO-ZETAa.s.Prague,CzechRepublicprozeta@prozeta.euwww.prozeta.euwww.tier5.cloud58LINBITHASolutionsGmbHVienna,Austriasales@linbit.comwww.linbit.com58
Klikněteapřidejtepoznámky
Zobrazení prezentace ve formátu HTMLMožnosti obrázku Přebarvení
Světlé 2
Nastavení
Průhlednost
40%
Jas
Kontrast
Resetovat úpravy
top related