object-based storage long liu 2010-10-23. outline why do we need object based storage? what is...
TRANSCRIPT
Object-based Storage
Long Liu2010-10-23
Outline
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
Outline
Background
• Existing enterprise storage infrastructures are feeling the strain
• the volume of data generated by many network-based applications continues to escalate
BackgroundTwo Technologies:
• Files:– Flexible data sharing– Secure
• Blocks:– High performance– Scalable
Blocks
Files
Comparison
Block Based Disk Object Based Disk
Operations: Read block Write block
Addressing: Block range
Operations:Create object Delete object Read objectWrite object
Addressing: [object, byte range]
Objects can be regarded as the convergenceof two technologies: files and blocks
Comparison
Object Interface
File System
Storage component
ApplicationsApplications
Logical Block Interface
File System
Traditional Storage Object-based Storage
Hard Drive Object-based Storage Device (OSD)
Comparison
File SystemStorage Component
Block I/O Manager
Storage Device
CPU
Applications
System Call Interface
File SystemUser Component
File SystemStorage Component
Block I/O Manager
Storage Device
CPU
Applications
System Call Interface
File SystemUser Component
Object Interface
Block Interface
Two changes :Object-based storage offloads the storage component to the storage deviceThe device interface changes from blocks to objects
(a)Traditional
model
(b)Object storage
model
Motivation
Objects
•Improved device and data sharing – Platform-dependent metadata moved to device
• Improved scalability & security– Devices directly handle client requests– Object security
• Improved performance– Data types can be differentiated at the
device• Improved storage management
– Self-managed, policy-driven storage– Storage devices become more autonomous
Outline
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
Object-based Storage
Object
OSD(Object-based Storage Device)
MDS(Metadata Server)
Object-based Storage
Object
OSD(Object-based Storage Device)
MDS(Metadata Server)
Object
•An object is a logical unit of storage— ID (Identification)— Application data— Metadata which includes block allocation
and length— Attributes that is accessible by users
•Objects have file-like methods— open, close, read, write
Object
•The root object -- The OSD itself•User object -- Created by SCSI commands from the application or client•Collection object -- A group of user objects, such as all .mp3•Partition object -- Containers that share common security and space management characteristics
Object
P1
P2
P3
P4
User Data
AttributesObject ID
Metadata
U1
User Objects (for user data)
Collection Objects
Partition Objects
Root Object(one per device)
OSD
Object-based Storage
Object
OSD(Object-based Storage Device)
MDS(Metadata Server)
Object Storage Devices
Intelligence with its own CPU, Memory, Disk, Network
Redistribution with specific strategy
Objects’ metadata management
File SystemStorage Component
Block I/O Manager
Storage Device
CPU
Applications
System Call Interface
File SystemUser Component
File SystemStorage Component
Block I/O Manager
Storage Device
CPU
Applications
System Call Interface
File SystemUser Component
Object Interface
Interface
Two changes :Object-based storage offloads the storage component to the storage deviceThe device interface changes from blocks to objects
(a)Traditional
model
(b)Object storage
model
Object Storage Devices
Object Storage DevicesExpect wide variety of Object Storage Devices
Disk array subsystem
Stores up to 5 TBs per shelf
2 SATA disks – 240/500 GB Highly integrated, single disk
4 Gbps pershelf to cluster
Orchestrates system activityBalances objects across OSDs
Object-based Storage
Object
OSD(Object-based Storage Device)
MDS(Metadata Server)
Metadata Server
Control the interaction between clients and OSDs
Client
Metadata Servers (MDS)
File System
Applications
Storage component
Object-based Storage Device (OSD)
Metadata Manager
Object Storage Devices (OSDs)
Client
Dataflow of Metadata
Outline
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
T10 OSD Commands(face)
CR
EA
TE
/RE
MO
VE
RE
AD
/WR
ITE
GE
T/S
ET
AT
TR
Object
10001110101100000011101100111011110001111000
..…User Data
Opaque
attributes(stored only)
Shared
attributes
(stored &
processed)
Attribute pages
OSD Target
Interface
OSD Client
OSD Interface
OSD Commands(Interface)
• Basic Protocol– READ– WRITE– CREATE– REMOVE– GET ATTR– SET ATTR
• Specialized– APPEND– CREATE & WRITE– FLUSH– LIST
• Security– Authorization– Integrity– SET KEY– SET MASTER KEY
• Groups– CREATE COLLECTION– REMOVE COLLECTION– LIST COLLECTION
• Management– CREATE PARTITION– REMOVE PARTITION– FLUSH PARTITION– PERFORM SCSI COMMAND– PERFORM TASK MGMT
very basic
space mgmt
attributes
shared secrets
• opaque• internal• shared
Storage Technology Today
• Direct attached storage (DAS)• Fabric Attached Storage (FAS)
-Network Attached Storage (NAS)
-Storage Area Networks (SAN)
1
2
3
4
Direct Attached Storage
BACKUP SERVER
TAPE
RAID
RAID
RAID
RAID
Windows
UNIX
LANWindows
A traditional Direct Attached Storage
model
Fabric Attached Storage
CLIENTS
SERVER
DATA
C/SFabric Attached
Storage
Network Attached Storage
Clients
File I/O
IP network
File server
Block I/OStorage area network
Block storage
This figure illustrates NAS being used to share files among a numberof clients.
The files themselves may be stored on a fast SAN
Storage Area Networks
Clients
Metadata
Data
Block-based storage devices
Storage area network
Management
Servers
This figure illustrates a SAN file system
The files themselves are stored on a fast storage Area to which the clients are also attached.
Security
NETWORK
Management
METADATA SERVERS
Attribute
Capability
CLIENTS
Data
Capability
Attribute
OBJECT-BASED STORAGE DEVICE
Outline
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
Status
• Industrial– Lustre– Panasas
• Academic
Status
• Industrial– Lustre– Panasas
• Academic
Lustre
• First open sourced system with object storage• High-performance parallel file system• Consist of clients, MDS and OST(Object
Storage Targets)
Lustre
NETWORK
MetadataData
Data Management
ManagementMetadata
METADATA SERVERCLUSTER
CLIENTS
OBJECT STORAGE TARGETS(OST)
Panasas
• High-performance file system• Consist of OSD, Panasas File System, MDS
Panasas
Key Object Storage FeaturesIntelligent space management in storage layer
ßMedia geometry aware placement
ßData aware prefetching, caching & recoveryEncapsulation of data and attributes
ßNative object interface, good programming model
ßStorage interpreted attributes for per file properties
Key Object Storage AdvantagesßRobust, shared access by many clients
ßScalable performance via an offloaded data path
ßStrong fine-grained end-to-end security
Panasas
• Clients are from Energy, Government, Finance, Manufacturing and Higher Education
Status
• Industrial– Lustre– Panasas
• Academic
A Design of Metadata Server Cluster In Large Distributed Object-based Storage
Motivation:
• Metadata server cluster maybe the bottleneck• Frequent metadata access and movement• Terrible load balance management
VoDServer
WebServer
DatabaseServer
E-mailServer
FileServer
Storage Network(Fibre Channel)
DataMDS
Cluster
Security
Metadata
Application Server Cluster
Object-based Storage Device Cluster
Object-based Storage System Architecture
A Design of Metadata Server Cluster In Large Distributed Object-based Storage
Application Servers
ApplicationFile Hashing Manager
Mapping Manager
Metadata Server Cluster
Hashing Partition
Logical Partition Manager
Metadata Server Backend
Common Storage Space
Figure 3. Hashing Partition
A Design of Metadata Server Cluster In Large Distributed Object-based Storage
Hashing Partition:
A total solution for – File hashing– Metadata partitioning – Metadata storage
A Design of Metadata Server Cluster In Large Distributed Object-based Storage
Pathname: /Dir1/Dir2/filename
Mapping Manager
Pathname HashingResult (i)
PathnameMetadata
&etc
Pathname HashingResult (i+1)
PathnameMetadata
&etc
Metadata Server Cluster
Logical Partitions
1 4
2
3
Figure 4. Metadata Access Pattern
①.Filename hashing②.Selecting MDS through Mapping Manager③ .Accessing metadata by pathname hashing result④.Returning metadata to application server
Hashing Partition
Mapping Manager
2
1 3Metadata Server Cluster
Logical Partitions
Common Storage Space
4
Figure 5. MDS cluster failover procedure
A Design of Metadata Server Cluster In Large Distributed Object-based Storage
Outline
• Why do we need object based storage?• What is object based storage?• How to take advantage of it?• What's the status of object based storage?• What can we do about it?
From Our Perspective
The EndThank you