the developing needs for e-infrastructures

Joint Information Systems Committee

The Developing Needs for e-infrastructuresProfessor John Wood, Chair, JISC Committee for the Support of Research


Neutron powder diffraction data rates (1950-2010) (4 orders of magnitude gain with ILL/ISIS alone)

D20(ILL)

GEM(ISIS)

D1a(ILL)

NEUTRON MOORE’S LAW

Complex detectors with advanced electronics 4

ord

ers

of

ma

gn

itud

e in

30

ye

ars

1983

1998

efficient large solid angle detectors...

…fast electronics

e-technologies (e-science)

• bringing the central facilities into the universities

detectors & advanced data acquisition • unique synergy within CCLRC


In practice: Its all about scale

Creation:

Examining the detector arrays on the MAPs spectrometer at ISIS


A One off experiment

Collection:

An ATSR image of Sicily with Mount Etna eruption; taken 24 July 2001


Data Deluge!

Estd Data Storage RAL to 2010

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

2003-04 2005-06 2007-8 2009-10

Year

Vo

lum

e (T

B)

CSE

BADC

E-SCI

Diamond

PP

External

Total (TB)

Cum Total (TB)

Capacity:

eg at RAL

20PB by 2010

1PB = 1015 Bytes

Billions of Floppys

Millions of CDs

Thousands of PCs

(today’s)


Curation – who is responsible? Curation:

Some STFC based Repositories

– The Atlas Datastore

– The British Atmospheric Data centre

– The CCLRC Data Portal

– The CCLRC Publications Archive

– The CCPs (Collaborative Computational Projects)

– The Chemical Database Service

– The Digital Curation Centre

– The EUROPRACTICE Software service

– The HPCx Supercomputer

– The JISCmail service

– The NERC Datagrid

– The NERC Earth Observation Data Centre

– The Starlink Software suite

– The UK Grid Support Centre

– The UK Grid for Particle Physics Tier 1A

– The World Data Centre for Solar-Terrestrial Physics

Atlas Datastore Tape Robot


The problem will grow

New large scale facilities are being planned and built around the world.

They will be run remotely and have to interact in real time with HPC simulations, each informing the other. What will be the role of the researcher once the experiment starts?

Data storage etc needs to be planned right at the start.

An example: XFEL in Hamburg


Schematic layout of a single pass XFEL

A new X-ray source is needed for

studies of new, of non-equilibrium states of matter

at atomic resolution in space and time


3rd Gen. SR

2nd Gen. SR

Laser Slicing

SPPS

Initial

H.-D. Nuhn, H. Winick

Pea

k B

rig

htn

ess

[Ph

ot.

/(s

· m

rad

2 ·

mm

2 ·

0.1

%b

an

dw

.)]

FWHM X-Ray Pulse Duration [ps]

Future

Future

ERLs

X-Ray FELs

InitialUltrafast x-ray sources will probe space and time with atomic resolution.

Peak brightness of pulsed X-ray sources

what do we do todayandwhat tomorrow?


The X-ray free-electron lasers

will provide coherent radiation

of the proper wavelength and

the proper time structure,

so that materials and the

changes of their properties

can be portrayed at atomic

resolution in four dimensions,

in space and time.

Diffraction pattern of 10 x 10 x 10 Au cluster

Fascination - FELs for hard X-rays


Schematic presentation of transition states in a chemical reaction

Take a movie of chemical reactions


single molecule crystal

with atomic resolution Lysozym

Oversampling: J. Miao, K.O. Hodgson and D. Sayre, PNAS 98 (2001) 6641-6645

Imaging of a single bio-molecule


ã Firmenam e (Referentennam e)28

Coulomb explosion of lysozyme (50 fs)Coulomb explosion of Lysozyme LCLS

Radiation damageinterferes with atomicscattering factors and

atomic positions

50 fs3x1012 photons/100 nm spot12 keV

R. Neutze, R. Wouts, D. van der Spoerl, E. Weckert, J. Hajdu: Nature 406 (2000) 752-757

t=0

t=50 fsec

t=100 fsec

Coulomb Explosion von Lyzosym


The VUV-FEL user facility at DESY


RF gun

FEL experimental

area

bypass

4 MeV 150 MeV 450 MeV 1000 MeV

undulators

collimatorbunch compressorLaser

M1 M2 M3 M4 M5 M6 M7

250 m

bunch compressor

VUV-FEL


XFEL Length ca. 3.3 km

PETRA

HERA

European XFEL Facility in Hamburg

phase I

phase II


XFEL: Office and Laboratory Building


1k x 1k system with 4 x 4 super modules


DAQ Challenge: 2D X-Ray Detector Systems

106 pixels per frame for one detector

O(400-500) frames per train (goal, likely will start with less)

10 trains per second (machine allows up to 30 Hz…)

With 2 Byte/pixel average rate 10 Gbyte/sec for one 2D detector!

Time between frames as short as 200ns buffering needed

600 s

99.4 ms

100 ms 100 ms

200 ns

LPD


Data Storage Issues

Assume:

3 x 1 Megapixel 2D Detector Systems

2 Byte/pixel

500 frames per train are read taken and read out

10 trains per second

Running year: 200 days = 4800 hours

Running efficiency: 10%

Good-frame efficiency/compression: 25% (is this realistic?)

~13 Pbyte/year (1 Petabyte = 106 Gigabyte =1015 Byte)


Today’ Situation at DESY

– SUN SL8500 Tape Robot

– Installed at DESY in Jan 07

– Up to 10,000 Cartridges

– Multi library capability

– Lifetime of about 5 to 10 years (matter of running costs)

– Up to 64 drives possible

– Currently 30 drives LTO3 24 data/6 backup

– LTO3 400 GB/Cart, 120 MB/s

10,000 LTO3 Cartridges: 4 Petabyte V.Gülzow et al.


DESY Storage Capacity Planning w/o XFEL

2007 2008 2009 2010 2011 20120

1000

2000

3000

4000

5000

6000

7000

8000

9000

Storage Requirements DESY 2007 - 2012

Petra 3 Disk

Tier II Atlas

Tier II CMS

HERA Disk

Petra 3 Tape

HERA Tape

Years

Am

ou

nt

Ter

abyt

e

Year V.Gülzow et al.


Technology Forecast – Storage at DESY

Year Rate Capability

[Gbyte/sec]

Storage Space

[Petabyte]

2009 1 3

2012 5 26

2016 40 200

• not a technology problem• money and manpower issues• to be determined:

• user behaviour• compression and accept/reject algorithms

• potentially critical: access to data!


Exciting times ahead for JSR!

Raising awareness

Training

New interrogation tools and approaches

Infrastructure

Integrating information from many sources

Waking up decision makers

A new way of doing research for many communities

Who is going to be responsible for the long term storage, curation and authentication of data?

the developing needs for e-infrastructures

Documents