object storage is awesome.. etc "project cloud" qtr meeting @ disney/abc
TRANSCRIPT
Object Storage is Awesome!
Q4 2015
( Truth or…. ? )
presented at the
Project Cloud Quarterly Meeting
Today’s (storage) Wonk… Paul EvansPrincipal Architect
• 20+ years in Media & Technology engineering
• Tours of duty with Lucasfilm, IBM, PayPal, Brocade, Fox, USC
• OEM-level Integration using Commercial & Open-Source components
AGENDA
OBJECT STORAGE: - WHAT DOES THAT MEAN? - BENEFITS & REPERCUSSIONS - TIERING STORAGE - VALID & INVALID USE CASES
OBLIGATORY ‘FRAMING’ SLIDEDATA IS GROWING!
(Garter, IDC, etc. all say it is so….)Note:‘Clouds’arehottoo
“a storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manage
data as a file hierarchy and block storage which manages data as blocks within sectors and tracks.”
OBJECT STORAGE
OBJECT STORAGE
Object Storage: Split MeaningTO
PH
ALF
HAL
F BO
TTO
M
Object Storage: Split MeaningTO
P
BOTT
OMACCESSING
STORED DATAPROTECTING STORED DATA
OBJ
ECT
STO
RAG
E:
T
OP
HA
LF GENERIC HTTP EXTENDED API
PUTGET
DELETE
S3AZURESWIFT (OPENSTACK)
RADOSWOSSCALITYAMPLISTORCLEVERSAFE
…OTHERS
OBJ
ECT
STO
RAG
E:
T
OP
HA
LF BENEFITS
LOOSELY COUPLED CLIENTS
RICH METADATA
SEAMLESS SCALE
SIMPLE PROTOCOL
OBJ
ECT
STO
RAG
E:
T
OP
HA
LF ANTI-BENEFITS
OBJECT FRAGMENTATION
…and….
SLOW ‘PER-CLIENT’
OBJ
ECT
STO
RAG
E:
T
OP
HA
LF ANTI-BENEFITS…MACHINE-FRIENDLY,
ORGANICS- FRIENDLY
NOT
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALF
Example: RAID-BASED PROTECTION
If Data Was Stored by Ducks….
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALF
Ex: OBJECT-BASED PROTECTION
If Data Was Stored by Ducks….
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALFERASURE CODING
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALF
+ GEO-SPREADERASURE CODING
the (private) Object Landscape
Amplidata
Ceph SWIFT Amplistor&La<us EMC Cleversafe Scality WOS
open open commercial commercial commercial commercial commercial
naEve(in-kernel)
viaiSCSIGateway
viaiSCSIGateway
viaiSCSIGateway
viaiSCSIGateway
integratedgateway
viaiSCSIGateway
naEve viaFileGTWY hybrid semi-naEve via
FileGTWY semi-naEve hybrid
modular modular rateless(fountain)
CauchyReed-
SolomonLubyCRS proprietary
(Reed-Sol)proprietary(Reed-Sol)
License
block
file
erasure
coding
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALFBENEFITS
SCALE!
EXTREME RELIABILITY
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALFANTI-BENEFITS
COMPLEXITY -> LATENCY
‘DIFFERENT’-> CONFUSION
Also: BEST PERFORMANCE = REPLICA
OBJ
ECT
STO
RAG
E:
BO
TTO
M H
ALFNOTABLE ‘OS’ EXCEPTIONS
(in the realm of performance…)
Custom OSS/OST Primarily HPC/Linux usage
LUSTRE
ANSI T10 OSD model Improved SMB access
PanFS
RADOS+CRUSH Object / File / Block
CEPH
Out-of-Band Metadata ‘StorNext on NFS’
pNFS
TIE
RIN
G D
ATA
SEE
TH
E BI
G P
ICTU
RE
Object Storage / API Cold StorageArchive
Virtualized Namespace
block storage
openstackhadoop
Scale OutStorage
RemoteCloud
Access
Direct
OBJECT STORAGE USESVA
LID• GLOBAL ‘SHARED’ ACCESS• LOW BIT-RATE STREAMING• ORIGIN SERVER (CDN)• BULK PROCESSING (TRANSCODING)• BULK STORAGE / STAGING SPACE
OBJECT STORAGE USES
SUB-
OTP
IMAL• REAL-TIME ACCESS
• HIGH-THROUGHPUT CLIENTS• PERFORMANCE EFFICIENCY• LOW-LATENCY APPLICATIONS USUALLY
In Summary…orSAN NAS& OBJECT
Imperial Envoy of Massive Persuasion
Light/Fast vs Work-at-Scale
