safe replication and data staging core services of the eudat cdi johannes reetz, rzg 2nd eudat...
TRANSCRIPT
Safe Replication and Data StagingCore Services of the EUDAT CDI
Johannes Reetz, RZG
2nd EUDAT conference
Rome, 29 Oct 2013
2
domain of registered data
individual value
preservationacquisitiongenerationdescription
enrichmentprocessingreductionanalysis
Data Stage-Out
publication
Safe Replication
community valueDat
a Sta
ge-In
society value
IdentifierService
EPIC Handle PID
4
Replication from repositories to data storages in different administrative domains
• (long-term) archiving and preservation
• optimize access for users from different regions
• bring data closer to powerful computers for data analytics
Typical policies triggered by Community Data Managers:• Replicate collection X from my repository to data centres A and B• Store the replica safely for N years• Check the integrity of the replica every M years
5
Transferring data from EUDAT storages to compute facilities
• reliable, efficient, easy-to-use tools to manage data transfers
• ingest data into the EUDAT domain of registered data
Each data centre (administrative domain) has a dedicated Handle namespace (prefix)
replica storages
repositories
6
EPICHandleService
http://cmon.eudat.eu
EPICHandleService
enabled EUDAT sites
replica storages
repositories
7
http://cmon.eudat.eu
EPICHandleService
GridFTP
8
CLARIN (MPI-PL/TLA)production
9
CLARIN (EKUT)production
10
CLARIN (CUNI/LINDAT)production
ENES (DKRZ/CMIP5)
11
production
12
EPOS (INGV)production
13
VPH (UCL)production
14
EPOS (ORPHEUS/ KNMI)work in progress
15
INCF (KTH)work in progress
16
DiXa (EBI)work in progress
Track 1 – Agenda – Session 1 17
14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz
14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari
14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers
14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak
14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney
15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni
15:35: B2SAFE Practical Policies, Willem Elbers
15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi
Track 1 – Agenda – Session 1
use cases in production
Track 1 – Agenda – Session 1 18
14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz
14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari
14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers
14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak
14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney
15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni
15:35: B2SAFE Practical Policies, Willem Elbers
15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi
use case in production
Track 1 – Agenda – Session 1 19
14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz
14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari
14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers
14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak
14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney
15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni
15:35: B2SAFE Practical Policies, Willem Elbers
15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi
Track 1 – Agenda – Session 1
practical policies and policy management
Johannes Reetz RZG / MPG
Claudio Cacciari CINECA
Willem ElbersMax-Planck-Institute for Psycholinguistics
Pavel StranakCharles University
Peter CoveneyUniversity College London
Stefan Zasada University College London
Giuseppe FiameniCINECA
Maria Francesca Iozzi SIGMA/University of Oslo
SAFE REPLICATION AND DATA STAGING - Speakers
20