High Performance File System Service for Cloud Computing

Kenji Kobayashi, Osamu Tatebe

University of Tsukuba, JAPAN

Towards Cloud File System

Compute Cloud physical machines (e.g. Amazon EC2)

Storage Clouddisk array (e.g. Amazon S3)

We propose file system service in Compute Cloud

Application modification required -REST, SOAP based protocol

✗

Low and Unstable Performance✗

Expensive Cost for constructing -High bandwidth Network between EC2 and storage

✗

Maintainability○Availability○

Gfarm File System

migration

replicationremote access

merely using local disk

Availability

Maintenancability✗✗

Availability

Maintainability○○

Towards Cloud File System (2)

Towards Cloud File System (3)

Construct File system service using stable HostOS on physical machines

VM

VM

VM VM

VM

HostOS HostOS HostOS HostOS HostOS

the HostOS is the baseOS for managing VM.

need to solve this problem to take advantage of local file access

stable

virtualized environment hides the underlying physical configuration

volatile

difficult to utilize file access locality

System Design

• Construct Gfarm file system on compute cloud– Host OS provides functionality of I/O server

• Create the physical configuration file dynamically – Take advantage of local access performance

gfsd

VM

Gfarmclient

HostOS

gfsd

VM

Gfarmclient

HostOS

gfsd

VM

Gfarmclient

HostOS

gfsd

VM

Gfarmclient

HostOS

gfsd

VM

Gfarmclient

HostOS

gfsd

VM

Gfarmclient

HostOS

Virtualized environment prevents to identifythe Host OS running on physically same node

Need some mechanism to identifythe physical configuration

System Design (2)

EucalyptusNC

gfsd

gfmd

Mapping File<instance id=“i-xxxxxx”> <vm_ip_addr> 172.16.xxx.xxx </vm_ip_addr> <hostOS_ip_addr> 172.16.xxx.xxx </hostOS_ip_addr>

･･･</instance>･･･

VM

Gfarmclient

EucalyptusNC

gfsd

VM

Gfarmclient

EucalyptusNC

gfsd

VM

Gfarmclient

･･･

Eucalyptus NC generatesmapping file dynamically

reply for message to access gfsd

real network

request metadata

HostOS HostOS HostOS

Inform the mapping to metadata server

Thank you for your attention.