Repositories, Workspaces

Workshop

organized by MPI PL, RZG, SARA, FZJDate: 20. - 21. September 2010 9:00

~25 participants from Finland, France, Germany ,

Italy, Netherlands, US

1. RZG2. practical information3. workshop agenda/intention

Welcome!

3

Foto: Artur Gerngross, Fa. FOTAG

LRZ

IPP

MPQ

MPE MPA

RZG

Science Campus Garching near Munich

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

IPP is one of the largest fusion research centers worldwide,an associate of the European Fusion Program

and the ITER organization

MPI for Plasma Physics (IPP)

W7-X Stellerator experimentASDEX UpgradeTokamak experiment ITER Tokamak experiment

Fusion Research at the

55

RZG computer hall

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

1969:IBM360/91 1979: Cray-1 1996/1998:Cray T3E/8161962:IBM 7090

2002/2003 IBM p690/1024

1999NEC SX-5/3C

seit May 2008IBM p575 Power6

seit Sep 2007IBM BG/P

Supercomputing at RZG since 1962 andHPC application support

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz 6

7

Cluster-Hostingmore than 20 Linux-Cluster from 15 Max Planck Institutes

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

FHI Berlin 36TF (SUN)(Prof. Scheffler)384 Knoten á 8 Cores (3072 cores), 4 Racks MPI für Physik Komplexer Systeme, Dresden

18 TF (SUN)(Prof. Moessner)ca 192 Knoten, 2 Racks (1536 cores)

MPI für Astrophysik 18 TF (Prof. Asplund)ca 192 Knoten, 2 Racks (1300 cores)

MPI für Astronomie Heidelberg 18 TFcirca 192 Knoten, 2 Racks (1008 cores)

MPI für Biochemie 18 TF (HP)circa 192 Knoten, 2 Racks

MPI für Chemie Mainz 4.5 TF (IBM) (Prof. Lelieveld)ca 48 Knoten, 0.5 Racks

MPI für Anthropologie Leipzig und MPI für Züchtungsforschung Köln(Prof. Paboo, Prof. Schulze-Lefert)grosses shared-memory System

IPP Garching 16 TF2 cluster, 1600 cores MPQ 336 cores

weitere Systeme für: MPE,CPFS,EVA, Biochemie

8Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

experiment andobservationaldata

simulationsdata

backup data

archive data

1389 TB (18%)

484 TB (6%)

2205 TB (29%)

890 TB (11%)

2518 TB (33%)2nd copy

Total

7486 TB (100%) Stand 13.09.2010

RZG tape robot Sept. 2010Capacity 14 – 40 PB (LTO3,4,5)

RZG data services: Backup/Archive, long-term archive present data in the TSM/HSM storage system

9

RZG data services: long-term archivedata security, availability, reliability Mirroring archive data mutually between RZG and LRZ

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

experiment data

Tape ArchiveRZG

Tape ArchiveLRZ

AFS (+ OSD)

GPFS 1 + TSM-HSM

HPC Simulations dataBackup/Archive Data

TSM Backup/Archive Servers

GPFS 2 + TSM-HSM

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz 10

Status 13.09.2010: 7.5 PB

RZG data services: growth of data in the long-term archive

11Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

→ Computer Centre of the Max Planck Society and IPP

→ High Performance Computing

→ HPC application support and visualisation

→ Compute-Cluster Hosting for Max Planck Institutes and Projekts

→ Experiment data aquisition

→ European und national Projects (DEISA, ELPA, ISAR, Atlas/Tier2, …..)

→ Data services: file systems, storage systems, backup/archiving, long-term archiving (50yr lifetime warranty for Max Planck Institutes)

RZG profile

12Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

20.9.2010 Repositories and Replication Title

9.00 Johannes Reetz (RZG) Welcome and Intention 9.15 Peter Wittenburg (MPI) Architecture Issues and Overview 9.45 Larry Lannom (CNRI) Abstract CNRI Model10.45 Coffee break  11.15 Alex Wade (MS) Business Approaches12.15 Malte Dreyer (MPDL) Generic API Principles13.00 Lunch Break IPP Cantina14.00 John Kennedy (RZG) Data Repositories within DEISA14.30 Kenneth Galluppi (NOAA) Federating climate, water and weather data repositories15.30 Willem Elbers (MPI) Replix project – motivation, goals, results16.00 Coffee break  16.15 Jean-Yves Nief (IN2P3) Generic policy rules and principles17.15 Daan Broeder (MPI) API Data Grids-Repositories18.00 End  19:30 Joint Dinner

Pizzaria “La Pergola” Münchner Str. 4, at U6 station „Garching Zentrum“http://www.lapergola-garching.de

 21.9.2010 Workspaces and Web Services

9.00 Peter Wittenburg Wrap up and Discussion of previous day10.00 Maurice Bouwhuis, Marc vd Sanden,

Daan Broeder (SARA, MPI)Workspaces in CLARIN NL

10.30 Morris Riedel (FZJ) Workspaces – Concept and functional aspects11.00 Coffee break  11.30 Johannes Reetz, John Kennedy (RZG) Workspaces – requirements and operational aspects

12.00 Thomas Zastrow, Erhard Hinrichs (U Tübingen)

WebLicht Application and Workspace Requirements

12.45 Lunch Break  13.45 Mike Papazoglou (U Tilburg) Generic Workflow Frameworks for WebServices14.45 Peter Wittenburg Wrap up and Discussion 16.00 End  

1313

Practical Information

1. WLAN available either via eduroam or free access (ssid: “D2 GAST”).2. Please write your name, institution (email address) on the list of participants. 3. Transport from IPP to the Munich airport:

please write your name and departure time on the transport request form. We will try to arrange a transportation with IPP transportation service.

4. We will have lunch at the cantina of the Max-Planck-Institut for Plasmaphysics, just 150m distance from the seminar room. Lunch to be paid directly.

5. Evening dinner takes place on Monday 7:30pm at the Pizzeria “La Pergola”, Münchner Str. 4 in Garching (http://www.lapergola-garching.de). You find the menu (in german only) under http://www.lapergola-garching.de/pespeise.htm

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

14Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

20.9.2010 Repositories and Replication Title

9.00 Johannes Reetz (RZG) Welcome and Intention 9.15 Peter Wittenburg (MPI) Architecture Issues and Overview 9.45 Larry Lannom (CNRI) Abstract CNRI Model10.45 Coffee break  11.15 Alex Wade (MS) Business Approaches12.15 Malte Dreyer (MPDL) Generic API Principles13.00 Lunch Break IPP Cantina14.00 John Kennedy (RZG) Data Repositories within DEISA14.30 Kenneth Galluppi (NOAA) Federating climate, water and weather data repositories15.30 Willem Elbers (MPI) Replix project – motivation, goals, results16.00 Coffee break  16.15 Jean-Yves Nief (IN2P3) Generic policy rules and principles17.15 Daan Broeder (MPI) API Data Grids-Repositories18.00 End  19:30 Joint Dinner

Pizzaria “La Pergola” Münchner Str. 4, at U6 station „Garching Zentrum“http://www.lapergola-garching.de

 21.9.2010 Workspaces and Web Services

9.00 Peter Wittenburg Wrap up and Discussion of previous day10.00 Maurice Bouwhuis, Marc vd Sanden,

Daan Broeder (SARA, MPI)Workspaces in CLARIN NL

10.30 Morris Riedel (FZJ) Workspaces – Concept and functional aspects11.00 Coffee break  11.30 Johannes Reetz, John Kennedy (RZG) Workspaces – requirements and operational aspects

12.00 Thomas Zastrow, Erhard Hinrichs (U Tübingen)

WebLicht Application and Workspace Requirements

12.45 Lunch Break  13.45 Mike Papazoglou (U Tilburg) Generic Workflow Frameworks for WebServices14.45 Peter Wittenburg Wrap up and Discussion 16.00 End  

1515

Intention of the Workshop

Repositories/Workspaces Workshop, 20-21.09.2010

Johannes Reetz

• How diverse is the landscape of repository systems and are there mature architectural solutions emerging?

• how can the repositories from diverse landscapes be federated and are there suitable layers and APIs available?

• how can safe replication be done in such diverse landscapes?

• is it a useful approach to control activities in federated repositories by declarative policy rules and if so what are typical policies?

• what do we need user workspaces for and what characteristics to they have in a distributed environment?

• what are the requirements with respect to workspaces from the perspective of distributed processing chains driven by community applications?

• are there generic frameworks for the execution of distributed processing chains and the deployment of web services?