RaleighFS | RaleighDBA b s t r a c t S t o r a g e L a y e r

What is a File-Systems

Is a Method of storing and organizing datato make it easy to find and access.

...to interact with an objectYou name it, and you say

what you want it do.

The Filesystem takes the name you giveLooks through disk to find the objectGives the object your request to do something.

Image taken from namesys Reiser4

What is a File-Systems

On Disk Format (...serialized struct)ext2, ext3, reiserfs, btrfs...

Namespace (Mapping between name and content)/home/th30z/, /usr/local/share/test.c, ...

Runtime Service: open(), read(), write(), ...

...A bit of History

Kernel Space

User Space

User Program

System Call Layer

FS 1 FS 2 FS 3 FS N...FS 4

Vnode/VFS Layer

Multics 1965 (File-System Paper)A General-Purpose File System For Secondary Storage

Unix Late 1969Sun Microsystem 19842010 ...Till Now, no significant changes

The File-System

You can specify what byte to start to read/write from,

and the number of bytes to read/write.

A file is something that tries to look like a sequence of bytes.

You can read the bytes, and write the bytes.

Cutting bytes out of the middle or the beginning of a file, and inserting bytes into the middle of a file, are not permitted!

pread(fd, buffer, nbytes, offset)pwrite(fd, buffer, nbytes, offset)

ftruncate(fd, length)

creat(path, mode)open(path, flags)

Metadata (ctime, mtime, mode, ...)

(Data Blocks)

(Block Pointers)

Decompose a File-System

Semantic Layer

User Request

ResolveSemantic Layer

(Path/Query to Key)

Lookup Key

MetadataSemantic Layer

Lookup Metadata from Key

Object Pointerfor Read/Write

Requests

...to interact with an objectYou name it, and you say

what you want it do.

For the end user this name has a meaning and this meaning should be captured by the Semantic Layer,

while the rest of the Storage Layer is not interested in the meaning of the name.

User defined name has generally a variable length and tends to be verbose, while the storage layer needs

something fixed size and short, to ensure a quick lookup. To do this, objects names are converted in keys that can be a simple hash of the name or something more elaborated.

Semantic Layer

The semantic layer takes names and converts them into keys,

the Storage Layer take keys and finds the objects

User Request

ResolveSemantic Layer

(Path/Query to Key)

Lookup Key

MetadataSemantic Layer

Lookup Metadata from Key

Object Pointerfor Read/Write

Requests

Operationscreate(): Create a new object, Unix place this object in parent directory object, Set Unix Stat, ...open(): Open specified object.lookup(): Lookup Key of specified object.

move(): Change name or location of specified object.unlink(): specified object, Unix remove this object from parent directory object.

Semantic Layeru n i x S e m a n t i c

Every objectmust be in one directory

root ‘/’ is the entry point

Parse Object Nametraverse each directory

check permissionand open it.

Internal nodes Leaf nodes (Stat/Meta data)Root node

A B+Tree can be usedto map Object Key

to its Metadata

Semantic LayerF l a t S e m a n t i c

Same Levelfor every Objects

No forced Hierarchy

Lookup item just by name

No Directory Traversal open(‘mytable’)

open(‘office-documents/stats’)

Object Layer

create(): Initialize object data structure for creation.open(): Initialize object data structure for open.close(): Uninitialize object data structure.

read(): Read specified object data.write(): Write specified data to object.append(): Append Data to object.remove(): Remove specified data from object.

truncate(): Truncate or extend object to specified length.inject(): Inject block data to a specified object.chop(): Remove block data from specified object.

An object contains your data

Different Data Types have different

methods and needs

MimicLanguages Typesset, dict, list, ...

Log Object (Append Only)

KV Object (Hashtable)

Set Object (Think at Dirs)

Flow Object (Write Anywhere)

Table Object (Database Table)

Record Object (C Struct)...

Operations

• read(offset, length)

• write(offset, length)

• inject(offset, length)

• remove(offset, length)

• truncate(size)

Extent list,Pointers to data... Insert/Remove

Block Every-Where

Like a regular ‘80s filebut with more flexibility

Flow Object

• read(index, n)

• append(name)

• remove(index)

• remove(name)

Keep trackof objects stored

(names)

Object-AObject-BObject-C

... table/userstable/addrs

...

Object-AObject-XObject-YObject-Z

...

Pages list,Object Names...

Semantic Layerdoesn’t guarantee

to keep Objects Names

Dir Object

Wait! Wait! Dir Object is just a Set!

• read(recno)

• write(recno)

• inject(recno)

• remove(recno)

• truncate(n)

RecNo ObjectExtent Record list,Pointers to data... Insert/Remove

Record Every-Where

Like Flow Objectbut with a fixed size

user defined structure

Metadata keep tractfields sizes and names

Device Layer

Different Layoutfor different types

for different workloads

Where data is Stored?Memory

Disk (Raid?)Somewhere (DFS)

Block AllocationBitmapExtents?

Operationsalloc(): Allocate a block (touch bitmap/space-map)dealloc(): Deallocate a block (touch bitmap/space-map)

read(): Read some data from diskwrite(): Write data on disk

insert(): Insert Key/Value to the B+Treeremove(): Remove Key/Value from the B+Treelookup(): Retrive Key Value from the B+Tree

BlocksFixed Size

Variable Size

Device Layerk e e p t r a c k o f B l o c k s

Choose your Block4k, 16k, 64M

What do you need?Small Variable Size Files (B+Tree)Large Variable Size Files (Extents)

(Data Blocks)

(Block Pointers)

Worst caseOne block

Best caseContiguous

‘Normal’ caseLarge or Tail

Internal nodes Extent nodes Raw Data (leaf/blob)Root node

Device LayerB a c k R e f e r e n c e s

why fsck takes the whole day?Who owns the block X?

Metadata (ctime, mtime, mode, ...)

(Data Blocks)

(Block Pointers)

Put a back Ref into Data blocks!Metadata (ctime, mtime, mode, ...)

(Data Blocks)

(Block Pointers)

RaleighFS Structure

Flat Unix Memory Files DiskFlow Set Map

RecNo Tablecreateopenclosesync

moveunlink

createopenclosesync

queryioctl

createopenclosesync

readwritealloc

dealloc

insertremovelookup

registerunregister

notifycreateopensync

RPC Server

RaleighFS

Observers

Objects Device LayerSemantic Layer

SeqMap

insertupdateappendremove

Semantic LayerObjects LayerDevice Layer

createopenclosesync

lookupkey

moveunlink

insertupdateappendremovequeryioctl

syncreadwrite

insertremovelookup

To interact with an Object you name it, and you say

what you want it do.

v52005-2010RaleighFS

allocdealloc

A b s t r a c t S t o r a g e L a y e r

Matteo Bertozzi

Semantic LayerObjects LayerDevice Layer

createopenclosesync

lookupkey

moveunlink

insertupdateappendremovequeryioctl

syncreadwrite

insertremovelookup

To interact with an Object you name it, and you say

what you want it do.

v52005-2010RaleighFS

allocdealloc

A b s t r a c t S t o r a g e L a y e r

Matteo Bertozzi

Q&A