srb-n-srm for hep data management (possible architectures for brainstorming)
DESCRIPTION
SRB-n-SRM for HEP Data Management (Possible architectures for brainstorming). Arun swaran Jagatheesan [email protected] San Diego Supercomputer Center. A picture is worth a 1000 KB. First, options/possibilities in the Over all Architecture. SRM. SRM. SRM. SRM. SRM. SRM. /…/text1.txt. - PowerPoint PPT PresentationTRANSCRIPT
San Diego Supercomputer Center
National Partnership for Advanced Computational Infrastructure
San Diego Supercomputer Center
National Partnership for Advanced Computational Infrastructure
University of Florida
SRB-n-SRM for HEP Data Management
(Possible architectures for brainstorming)
Arun swaran Jagatheesan
San Diego Supercomputer Center
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 2
A picture is worth a 1000 KB
First, options/possibilities in the Over all Architecture
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 3
Distributed SRMs
/…/text1.txt /…//text2.txt
SRM SRM SRM SRMSRM SRM SRM
/txt3.txt
SRM
University in UK (B) Organizations in Asia (C & D)
National Lab in US (A)
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 4
Global Logical Namespace Needed
National Lab in US (A)
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
University in UK (B) Organizations in Asia (C & D)
/home/arun.sdsc/cms/home/arun.sdsc/cms/text1.txt/home/arun.sdsc/cms/text2.txt/home/arun.sdsc/cms/text3.txt
Logical Namespace: Organization of data, meta-data and storage (This logical view –
need not be same as physical view of data sources )
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 5
Global Logical Namespace Needed
National Lab in US (A)
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
University in UK (B) Organizations in Asia (C & D)
If we use SRB for this Global Logical Namespace.. .. (next slides)
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 6
Using Zone SRBs
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
SRB Zone for National Lab in
US
One or more SRB Zones in
Asia
One SRB Zone for all UK universities
Global Logical Name space
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 7
Possible Options for Zones
• Each Organization is a Zone with multiple SRMs• One of more organizations make a Zone based
on geography and administrative factors• Each country is a Zone
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 8
A picture is worth a 1000 KB
Second, Access
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 9
Do we want XML/SOAP for control?
Services (SOAP-based) to manage and operate on the logical namespace
of data and storage resources. Of course, this is only for control
channel and not data movement channel
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 10
Access Option1: SRM still the interface
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
SRB Zone for National Lab in
US
One or more SRB Zones in
Asia
One SRB Zone for all UK universities
Global Logical Name space
SRM SRMSRM
SRM
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 11
Access Option1: SRM still the interface
+ Existing applications in SRM can be preserved
+HEP community already aware of SRM
+SRM will emerge as the Grid Storage Management standard in GGF (so useful for us to follow the standard thingy)
– SRM interface does not deal with a global logical namespace
– Additional functionalities to manage or take advantage of this global logical namespace would be lost
– Bulk operations are essential for data grid management
– Zone to Zone (Peer to peer)
– Meta data
– SRM is NOT the Grid File System standard at GGF
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 12
Access Option2: SRB as the interface
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
SRB Zone for National Lab in
US
One or more SRB Zones in
Asia
One SRB Zone for all UK universities
Global Logical Name space
SRB SRBSRB
SRB
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 13
Access Option2: SRB as the interface
– Existing applications might have to be modified
–HEP community has to be educated about the use and advantages of SRB (community resistance also)
–Does it directly support storage space management NOW?
+SRB interface deals with a global logical namespace of logical resources and logical data
+Additional functionalities to manage or take advantage of this global logical namespace is available+Bulk Operations are essential for
data grid management
+Zone to Zone (Peer to peer)
+Metadata
+SRB could influence the Grid File System standards at GGF
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 14
Access Option3: GGF Grid File System Interface
/…/text1.txt /…//text2.txt /txt3.txt
SRM SRM SRM SRMSRM SRM SRM SRM
SRB Zone for National Lab in
US
One or more SRB Zones in
Asia
One SRB Zone for all UK universities
Global Logical Name space
GFS
GFSGFS GFS
San Diego Supercomputer CenterGrid Physics Network (GriPhyN)
University of Florida 15
Access Option3: GGF GFS as the interface
– DOES NOT EXIST. We can NOT wait.
–Our limited cycles might be used in just working for the GGF standard – rather than making this interface for us to work
–What if we can not influence the standard creation (?) or its irrelevant for us
+It is going to be the standard. So would possibly end up having multiple implementations (commercial and academic)
+This community of different organizations and countries could influence the standard creation based on real needs
+Experts from other communities would also suggest / help in the design of this interface