institutions - sdm home · 2008-11-24 · institutions organizations srm/irods-srb dcache fermilab...
TRANSCRIPT
A. Sim, CRD, L B N L 1 SC 2008
SRMs
institutions
organizations
SRM/iRODS-SRB dCache
Fermilab
A. Sim, CRD, L B N L 2 SC 2008
A. Sim, CRD, L B N L 3 SC 2008
A. Sim, CRD, L B N L 4 SC 2008
A. Sim, CRD, L B N L 5 SC 2008
A. Sim, CRD, L B N L 6 SC 2008
A. Sim, CRD, L B N L 7 SC 2008
MSS
Storage Resource Manager
network
client Client (command line)
... Client’s site
... Disk Cache
Disk Cache
Site 2 Site 1 Site N
Storage Resource Manager
Disk Cache
Storage Resource Manager
Client Program
Disk Cache
Disk Cache
...
Storage Resource Manager
Disk Cache
Disk Cache
...
Uniform SRM interface
A. Sim, CRD, L B N L 8 SC 2008
• Non-interference with local policies • Advance space reservations
• Dynamic space management
• Pinning file in spaces
• Support abstract concept of a file name: Site URL
• Temporary assignment of file names for transfer: Transfer URL • Directory Management and ACLs
• Transfer protocol negotiation
• Peer to peer request support
• Support for asynchronous multi-file requests
• Support abort, suspend, and resume operations
A. Sim, CRD, L B N L 9 SC 2008
A. Sim, CRD, L B N L 10 SC 2008
A. Sim, CRD, L B N L 11 SC 2008
Slide courtesy: Jan van Eldik Giuseppe Lo Presti Shaun De Witt
A. Sim, CRD, L B N L 12 SC 2008 Slide courtesy: Timur Perelmutov
A. Sim, CRD, L B N L 13 SC 2008
Slide courtesy: Maarten Litmaath Flavia Donno Akos Frohner David Smith Jean-Philippe Baud
A. Sim, CRD, L B N L 14 SC 2008
A. Sim, CRD, L B N L 15 SC 2008
• DPM Name Server Namespace, Authorization, file location
• Disk Servers Physical files Direct data transfer from/to disk server (no bottleneck) Supports rfio, gridftp, xrootd and http(s)
• Supports SRM SRM-1.1, 2.2 (since April, 2007)
• Support DICOM backend for Biomed VOs Encryption of DICOM files on the fly + local decryption Use of GFAL and Hydra to get and decrypt the file
A. Sim, CRD, L B N L 16 SC 2008
Slide courtesy: Luca Magnoni
A. Sim, CRD, L B N L 17 SC 2008
Cache server (+gridftp server)
Core
Cache repository
SRB+DSI
File catalog
SRB/gridftp
Gridftp/management API Gridftp/management API
SRM API
File transfer (gridftp) File transfer (gridftp)
Web Service
Data server management
User
Hostname: fct01.grid.sinica.edu.tw The end point: httpg://fct01.grid.sinica.edu.tw:8443/axis/services/srm Info: Cache server (gridftp server) and SRM interface
Hostname: t-ap20.grid.sinica.edu.tw
Info: SRB server (SRB-DSI installed)
SURL
Gridftp management commands Return some
information
TURL
Slide courtesy: Fu-Ming Tsai Wei-Lung Ueng
A. Sim, CRD, L B N L 18 SC 2008
Clients
CASTOR dCache
dCache
Fermilab
Disk
BeStMan
StoRM
DPM
SRM/iRODS
SC2008 Demo: Interoperability of 6 SRM implementations at 12 Participating Sites
A. Sim, CRD, L B N L 19 SC 2008
SRM
Disk Cache
DISK CACHE
Client Job Gate Node
Worker Nodes
Disk
Disk
Disk
Disk
Client Job
Client Job
Client Job
Client Job submission
Fermilab
SRM/iRODS
dCache
dCache
dCache
SC2008 Demo: Analysis jobs at NERSC/LBNL with 6 SRM implementations at 12 participating sites
A. Sim, CRD, L B N L 20 SC 2008
A. Sim, CRD, L B N L 21 SC 2008
In alphabetical order
A. Sim, CRD, L B N L 22 SC 2008
• Storage Resource Management – essential for Grid • OGF Standard GFD.129 (Apr. 15, 2008)
• Multiple implementations interoperate • Permit special purpose implementations for unique products • Permits interchanging one SRM product by another
• Multiple SRM implementations exist and are in production use • Particle Physics Data Grids
• WLCG, EGEE, OSG, … • Earth System Grid • More coming …
• Combustion, Fusion applications • Structural biology, medicine
A. Sim, CRD, L B N L 23 SC 2008
• SRM Collaboration and SRM Specifications • http://sdm.lbl.gov/srm-wg • Developer’s mailing list: [email protected]
• BeStMan (Berkeley Storage Manager) : http://datagrid.lbl.gov/bestman
• CASTOR (CERN Advanced STORage manager) : http://www.cern.ch/castor • dCache : http://www.dcache.org
• DPM (Disk Pool Manager) : https://twiki.cern.ch/twiki/bin/view/LCG/DpmInformation
• StoRM (Storage Resource Manager) : http://storm.forge.cnaf.infn.it
• SRM-SRB : http://lists.grid.sinica.edu.tw/apwiki/SRM-SRB
• Other info : [email protected]
SRMs
institutions
organizations
SRM/iRODS-SRB dCache
Fermilab