ONLINE HIERARCHICAL STORAGE MANAGER FOR LINUX

Ottawa Linux Symposium 2009

Who am I ?

Greg Freemyer CTO and Litigation Triage Solutions Specialist at

The Norcross Group Using UNIX since 1983, and have an interest in

storage systems HP certified Master SAN Architect Participate in several kernel mailing lists and try

to help out on kernelnewbies I did not write any of OHSM, but I became an

advisor to the project after becoming aware of them via the kernelnewbies mailing list

OHSM Team

The ohsm team was formed to compete in various university competitions University of Pune, India

The team comprises of currently six members Project Mentor- Sandeep Kumar Sinha, NetApp, India Students - Rishi Bhushan Agrawal, Rohit Kumar Sharma, Rohit Vashist, Sneha

Hendre, Vineet Agarwal Competition results

Third Prize at Cognizance’ 09 held at Indian Institute of Technology, Roorkee. Consolation prize at Techkriti’ 09 held at Indian Institute of Technology,

Kanpur. First Prize at INNOVATION 2k9 held at Cummins College of Engineering, Pune. Second Prize at Jishin’ 09 held at Bharti Vidyapeeth College of Engineering for

Women, Pune. Third Prize at Techno Dream’ 09 held at Maharashtra Academy of

Engineering, Alandi.

Why OHSM ?

Extremely large cost ratio between low-cost consumer grade storage to high-end SSD class storage Bare SATA ~ $100/TB     Bare Intel SSD ~

$5,000/TB* Enterprises need online relocation of data

between storage tiers Enterprise Application Security Audits make

moving data from one logical location to another difficult and expensive

No open source solution

* Based on $750 for a 160GB Intel SSD on July 9, 2009

What is OHSM ?

File System Scanner

TRIGGERRelocation Policy

Dest = 3

Home= 1

Tier 1

Tier 2

Tier 3

Home = 3Dest = 0Dest = 0

OHSM Goals ?

Leveraging Hybrid aggregates (volumes) Policy based initial placement of files Online relocation of data between tiers Leverage user space, existing ABIs, and

existing kernel functionality Minimal Patches to existing filesystem(s) Kernel Module isolating OHSM kernel logic Mainline kernel inclusion

Complexity and Challenges

Relocating open files Keeping file system online Ranged block allocation Getting device mapping from device

mapper

Status of project

Prototype implementation on ext2 with both kernel and user space code

Design documents and full source available for ext2 prototype available for download

ext3 testing in progress ext4 is in the architectural discussion

stage

Prototype Architecture

High Level Architecture

Detailed Architecture

User Space Components

XMLParser

Device Mapper Library

OHSMAdministrat

or

Device Topology

OHSMUser

Interface

OHSM Kernel Driver

Tier Block Range Map

Meta Data

File System Scanner

Policies

File System

Sample policy file in XML

<DEVICES>

<DEV_TIER_INFO>

<NR_TIERS>3</NR_TIERS>

<NR_DEVICES>6</NR_DEVICES>

</DEV_TIER_INFO>

<DEV_TIER>

<TIER>1</TIER>

<DEVICE>/dev/md4</DEVICE>

<DEVICE>/dev/md5</DEVICE>

<TIER>2</TIER>

<DEVICE>/dev/md3</DEVICE>

<TIER>3</TIER>

<DEVICE>/dev/md1</DEVICE>

<DEVICE>/dev/md2</DEVICE>

<DEVICE>/dev/md6</DEVICE>

</DEV_TIER>

</DEVICES>

Policy File Generator : Allocation

Policy File Generator : Relocation

Policy File Generator : Tier Device

Configuration Files

/etc/ohsm/dtd_alloc.dtd /etc/ohsm/dtd_reloc.dtd /etc/ohsm/dtd_device.dtd /etc/ohsm/ohsm.conf

User space / Kernel APIs

Enable/Disable OHSM ioctl services Set Allocation and Relocation Policies Set Tier Device Mappings Enable/Disable Relocation Policy ( Based

on Relocation rule ID or Rule No) Triggering relocation Get Tier Status  (No. of allocated/free

blocks)

Ext4 implementation

New Architecture design for ext4 supporting per-inode preallocation and per-CPU locality preallocation

Ted Tso's Dec. 8, 2008 ext4 post 1 - 3 new ioctls: Blocks NOT to allocate from – global Preferred Block range to Allocate from -  Inode

specific replace blocks with donor blocks - Inode

specific

1 - http://markmail.org/message/qp7zjhhdzxum7rfn

Ext4 Implementation….

Akira Fujita Patches EXT4_IOC_ADD_GLOBAL_ALLOC_RULE 2 – rfc EXT4_IOC_ADD_INODE_ALLOC_RULE 2 – rfc EXT4_IOC_MOVE_EXT 3 - in 2.6.31-rc series

Open/Create Hooks Home and Destination Tier, where to store?

 

  2 - http://markmail.org/message/yigzopzx2gkdjyw3

  3 - http://markmail.org/message/a43cgtefhhz45slc

What is the OHSM Project ?

Effort to build platform for HSM on Linux Idea came when thinking of hybrid

aggregates (SSD + Rotational) Its hosted at code.google.com/p/fscops

and ohsm.sourceforge.net Open to all

Future of OHSM ?

A free Open Source Hierarchical Storage Manager for Linux framework

Ext4 support Submission of patches to mainline Allocation based on directories Analytical Engine to trigger relocation Support for extending FS when Logical

Volume is extended Allocating inodes from optimum tier based on

policy

?Questions