1

DataTAG-WP4 and GLUEwww.cern.ch/datatag

(mainly from A.GhiselliII INFN-Grid workshop)

Mirco Mazzucato Gridstart meeting at HPC 2002

24-6-2002 Cetraro

2

DataTAG-WP4

main focus:Interoperability between DataGrid in EU and GriPhyN and PPDG in US

US partnership: iVDGL project

3

Common EU-US activities: GLUE

• Acquire and define software, configuration guidelines and policies to allow Middleware Services developed by the participating Grid Projects to Interoperate.

• Results must be independent of the particular Data Grid Architecture implementation. – But to succeed will need agreement on Principles

• What does Interoperate mean ?

– Someone in US/EU can run on the EDG/US Grid using their US/EU obtained credentials.

– Someone in EU/US can submit a Job in the EU/US that will be scheduled on Resources that span EU and US sites.

– Someone in US/EU can move Data from/to US and EU sites without specific knowledge of which continent the Site is on – the knowledge of the Site Resources, Performance, and Policies is sufficient.

4

The interoperability issues• 1) Certificates. This issue is almost closed. It should be made operational.

– Need to agree on management of revocation lists – Distribution of Certificate files not consistent or automated2) GSI security. Some people do not consider the error reporting satisfactory

for an LCG/EGEE production infrastructure. Globus is aware of this problem. A discussion is needed and probably an effort should be set to follow and define fully the requirements of the improvement plan

• 3) Authorization and VO management. A joint subproject has been set up. A work plan for a September recommendation should be put in place.

• 4) Schema. Here the work plan exists, the work is progressing well. CE schema agreed. Working now on SE.

• 5) GIIS structure and hierarchies. This sub-project is on the GLUE pipeline.• There is the general issue of the different vision of the information system

based on LDAP or on R-GMA. Probably this could be another sub-project but EDT would leave to Grid projects to decide.

5

The interoperability issues(cont)• 6) Scheduling, use of JDL and experiment interface. Regular meetings are

taking place between EDG WP1 and Condor/ PPDG people. This should be acknowledged as an official GLUE activity and produce a recommendation for September.The issue is to allow experiment to have a unique interface e.g.: DG-submit vs Globus run or Condor-submit….

• MOP vs EDG Resource Broker• 7) Data management.• A good collaboration on these topics already exist between EDG WP2 and

Globus/PPDG teams. Again this collaboration should be transformed in an official GLUE activity and make a common recommendation on the usage of GDMP, Replica Catalog and Replica Manager

• 8) Packaging. • LCG/Grid projects should have common release, packaging and installation

tools to favour integration and deployment• The interest for these topics goes beyond the Grid projects and the release of

the LCG middleware since it includes the software of the LHC experiment. An LCG application effort is being initiated by Torre Wenaus to define a common LHC solution. The time frame for this activity is not yet clear. Other Grid projects should be integrated

• Now EDG has RPM and LCFG and US (VDT) has PACMAN• There is the need to distribute EDG components in US and viceversa

6

DataTAG projectWP4.1-3 service interoperability

WP4.4 application interoperability

WP4.1 activity report(resource discovery)

7

GLUE test layout for CE,SEPeople

DataTAGEU Schema Test and layout setup:

Cristina Vistoli (coordinator)EDG

Resource Broker integration : Massimo SgaravattoInformation provider : Olof Barring, (iVDGL Globus)SE schema definition : John Gordon , (iVDGL Arie

Shoshani)GRIT A TestBed solely for the purposes of testing interoperability across the Grid Projects – without impacting progress of each End User, Project or Experiment. Set up by EDT and iVDGL

8

CE Schema implementation status

• Schema logical structure is stable

9

CE Schema implementation status

• Ongoing LDAP implementation from both EU and US side• The EU prototype one ready for the end of next week• The US one ready in a couple of weeks• Main goals:

take the best ideas from bothand agree on

a common LDAP schemaa common LDAP DIT

that will be delivered with the next Globus Toolkitand adopted by EDG (Release1.4?)

10

DataTAG project

WP4.2 and GLUE program(security authorization)

11

GLUE – VO Authorization Management

Basic Steps (GLUE phase I)– Agreeing on supported CAs (Certificate Authorities)

– User and VO (Virtual Organization) authorization based on the subject of the certificates (following Globus implementation).

– Access Authorization on the CE (for ex. gridmap file Globus based, evolving to LCAS in phase III (Local Community Auth. Server) )

– CE-User association, readable by grid tools or applications (for ex. Glue CE schema with the accepted VOs id. published).

12

Useful Mechanisms• GLUE phase I

– Securing VO server implementation (eg. more restrictive access to LDAP server).

– Let users specify VO membership during authentication (multiple VO membership management).

– Simple VO management mechanism (web user interface for VO registration)

• GLUE phase III– CAS/LCAS (aggregate user management)

13

DataTAG experiment grid integration plan and GLUE

tasks 4.3-4.4

Flavia Donno, INFN-Pisa

14

Outline

• Goals• People and HW resources• Working plan with experiments (ALICE, ATLAS, CDF,

CMS, D0, VIRGO)• Organization and Conclusions

15

GoalsGoals:

Interoperability between EU and US Grid services from DataGrid, GriPhyN, PPDG and in collaboration with iVDGL, for the HEP applications as seen from these(LHC experiments as

reference but open to others)Difference: Grid projects manpower are part of experiments in US

Steps:Produce an assessment of interoperability solutionsProvide test environment to LHC Applications to extend existing use-cases to test interoperability of the grid components Provide input to a common Grid LHC architecture Provide input to plan EU-US Integrated grid deployment for experiments

17

Working plan with experiments - 2002

Plans established with Alice, Atlas, CMS, main Plans established with Alice, Atlas, CMS, main focus for EDT, but also focus for EDT, but also D0,D0, CDF, Virgo (/LIGO) CDF, Virgo (/LIGO) through unfunded effortsthrough unfunded efforts

CMS (and CDF/D0 US counterpart) already CMS (and CDF/D0 US counterpart) already involved in integration plans (LIGO is coming).involved in integration plans (LIGO is coming).

First integration attempts for CMS on-going. EU First integration attempts for CMS on-going. EU involved in job submission (Impala interfaced with involved in job submission (Impala interfaced with EDG-RB), US involved in data management (Virtual EDG-RB), US involved in data management (Virtual Data) and interactive jobs. Data) and interactive jobs.

First integration attempt for Virgo on going.First integration attempt for Virgo on going.

18

Working plan with experiments - 2002

Packaging and Distribution GroupPackaging and Distribution GroupStudy core HW and software to be used.Study core HW and software to be used.

Study existing distribution tools. Experimental distributionStudy existing distribution tools. Experimental distribution

Recommendation for convergence.Recommendation for convergence.

Job Scheduling and Management Services GroupJob Scheduling and Management Services GroupStudy of middleware used (EDG/WP1,Study of middleware used (EDG/WP1, DAGMAN, Condor-G, etc.)DAGMAN, Condor-G, etc.)

Experimental integration activities (CMS/Impala-Boss, Alice/AliEn, Experimental integration activities (CMS/Impala-Boss, Alice/AliEn, CDF/SAM) CDF/SAM)

Data Management GroupData Management GroupStudy of middleware used (EDG/WP2, GDMP, GridFTP, etc.)Study of middleware used (EDG/WP2, GDMP, GridFTP, etc.)

Experimental integration activities (ATLAS/Magda, CDF/SAM Experimental integration activities (ATLAS/Magda, CDF/SAM Virgo/GDMP)Virgo/GDMP)

5 areas of interoperability work:5 areas of interoperability work:

19

Working plan with experiments - 2002

MetaData Catalogues GroupMetaData Catalogues GroupStudy of requirements and middleware (EDG/Spitfire, PPDG/Virtual Data Study of requirements and middleware (EDG/Spitfire, PPDG/Virtual Data Catalogue)Catalogue)

Prototype activities (Alice, CMS)Prototype activities (Alice, CMS)

Experiment integration support groupExperiment integration support groupFollowing experiment activities outlining special needsFollowing experiment activities outlining special needs

First integration testsFirst integration tests

5 areas of work:5 areas of work:

20

DataTAG-WP4 and GLUE programphase I

• Task 4.1 Resource Discovery – Common resource schema definition (CE,SE)– MDS schema implementation – MDS-GIIS hierarchy test on GLUE layout

• Task 4.2 Security Authorization– VO based authorization management– Requirements definition for CAS

• Task 4.3-4.4, high level grid services interoperability – Common Packaging tools for Grid kits– Scheduling jobs on GLUE test layout

21

DataTAG-WP4 and GLUE program phase II

• Task 4.1 Resource Discovery – Test of EDG-GMA(Grid Monitoring Arch.)

information service • Task 4.2 Security Authorization

– Common CAS implementation• Task 4.3-4.4, high level grid services interoperability

– Data movement infrastructure– Scheduling and data replication

22

DataTAG-WP4 and GLUE program phase III

• Task 4.1 Resource Discovery – Others information services, web services?

• Task 4.2 Security Authorization– Common CAS test

• Task 4.3-4.4, high level grid services interoperability – Complete interoperable computational service

validation