Dominik Stoklosa

Poznan Supercomputing and Networking Center, Supercomputing Department

EGEE 2007 Budapest, Hungary, October 1-5

Workflow management in Remote Instrumentation Infrastructures –

e-VLBI experiences

Introduction to the e-VLBIIntroduction to the e-VLBI

EXPReS projectEXPReS project

PSNC in EXPReS - FABRIC PSNC in EXPReS - FABRIC

System designSystem design

Managing data flowsManaging data flows

Outline

Introduction to the e-VLBIIntroduction to the e-VLBI

VLBI is a technique, in which physically independent and widely

separated radio telescopes observe the same region of sky

simultaneously, in order to generate very high-resolution

continuum and spectral-line images of cosmic radio sources

Telescopes are usually separated by thousands of kilometres

Data from each telescope are digitally sampled and stored locally,

using high-capacity magnetic tape systems and magnetic disk-array

systems

Data are sent and correlated at the central point (JIVE)

The total flow of data into the central processor is approximately 10-

100 Terabytes per single observation, after processing this is reduced

to 10-100 Gbytes.

Radio TelescopesRadio Telescopes

Arecibo, ChileOnsala, Sweden

RT4, Poland

Westerbork / Very Large ArrayWesterbork / Very Large Array

408 Mhz

optical

1.4 Ghz

Radio / OpticalRadio / Optical

Introduction to EXPReSIntroduction to EXPReS

EXPReS – the objective is to create a production-level

“electronic” VLBI (e-VLBI) service, in which the radio telescopes

are reliably connected to the central data processor at JIVE via a

high-speed optical-fibre communication network.

Project Details

Three years, started March 2006

International collaboration

Funded at 3.9 million EUR

FP6, Contract #026642

Introduction

EXPReS partnersEXPReS partners

19 partners, 21 telescopes, 6 continents

PSNC in EXPReSPSNC in EXPReS

EXPReS – a Real-time e-VLBI Radio Telescope

- JRA1: Future Arrays of Broadband Radio-Telescopes

on Internet Computing (FABRIC)- Grid – VLBI collaboration- Grid Workflow management

- Grid Routing

Creating solution for incorporating Grid resources for

distributed correlation using existing infrastructure.

Once upon a time (1)Everything was slow Telescopes collected data on tapes Sent via postal mail

Hard drive arrays slightly improved the situation

The entire cycle could easily require 6 months or more

Once upon a time (2)

Hardware correlator; the EVN MkIV data correlator at JIVE dedicated, purpose designed/built hardware a super computer; ~100 T ops/sec

Today / In the near future

Data can be transferred over the network

Each stage of the process can be speeded up

GRID resources

Software correlator

e-VLBI - electronic VLBI

System design – data flows (1)

System design – data flows (2)

WFM – phase 1

Definition of radio telescopes – automatically based on theobservation schedule

WFM – phase 2

Definition of file servers (each file sever is responsible for capturing data from RT)

WFM – phase 3

Definition of correlation nodes and data flows between components

WFM – properties

Definition of resource properties

Remote Instrumentation in Next Gen GridsRemote Instrumentation in Next Gen Grids www.ringrid.euwww.ringrid.eu

RINGrid – SSA project funded under 6th FP

—Partners from UK, Austria, Greece, Italy,

Romania, Bulgaria, Mexico and Brazil

RINGrid objectives:

—Identification of instruments and user communities, definition of requirements

—Trends definition and recommendations for designing next-generation Remote Instrumentation Services

—Promoting equal access to European e-Infrastructure opportunities

http://www.ringrid.eu/

Thank you for your attention

http://www.expres-eu.org/d.stoklosa@man.poznan.pl

EXPReS is made possible through the support of the EC, 6th FP, Contract #026642