the astronomical virtual observatory€¦ · the astronomical virtual observatory françoise genova...
TRANSCRIPT
The astronomical Virtual The astronomical Virtual
ObservatoryFrançoise Genova
OGF-Europe Community Outreach Seminar
Digital Repositories - Interoperablility Using Grid Technologies
OGF23 - 5 June 2008, Barcelona, Spain
• Sustainability is a central issue
• Talk centered on challenges & lessons learnt from building the astronomy data services and the Virtual Observatory to give seamless access to them
• Brief description of the astronomical Virtual Observatory
• Different points of view on requirements and lessons learnt
– Users
– Data Centres
– VObs project
– Funding bodies
The astronomical Virtual
Observatory (1)
• Heterogeneous, distributed ‘data’ services: archives of observations, value-added data bases, tools, bibliographic data (including e-journals), simulation data
• Most data freely accessible after proprietary period
• The Virtual Observatory concept
Seamless and transparent query of data centres
New analysis and visualisation tools
A standard structure for data centres to publish
their data and services
The Astronomical Virtual
Observatory (2)
• A thin interoperability layer
• A data and service grid, incrementally made available for community usage
• An international endeavour from the beginning
• International Virtual Observatory Alliance
– An alliance of very diverse national projects, each depending of its own national context
– Defines a common roadmap and discipline-specific interoperability standards
Europe (incl. ESA,
ESO + 6 partners)
China
India
Canada
SpainItaly
Armenia
FranceFranceGermany
Hungary
Japan
Korea
USA
Russia
UK (AstroGrid)Australia
The IVOA Working Groups
• Standardization process adapted from W3C
• Working Groups– Grid & Web Services (assesses usage of Grid technologies)
– Resource registry (OAI compliant but much more information for retrieving resources)
– Data modelling
– Query language
– Data Access layer
– Semantics
• Interest Groups: Applications; Theory; Data Curation & Preservation, OGF Astro-RG
Data available at selected point are highlighted in tree
Genova, The astrophysical Virtual
Observatory
Field of view outlines are plotted automatically
Image metadata – provenance and characterisation
AVO prototype based on CDS Aladin
Data Centre AlliancePopulates the VOwith data and servicesFP6 Coordination Action tohelp data providers andgather feedback
The VObs components
Euro-VO
Best effort alliance
8 partners
Technology CentreDistributed• FP6 VO-TECHInfrastuctureDesign Study• ESA-VO (et al.)
Facility CentreSupport to usersRegistry
FP7 I3 EuroVO-AIDA
Scientists (1)
• They want to do their science
• They want to be able to reuse data
• But they also want to get new instrument and direct
support to their science programssupport to their science programs
• Not so easy to put some money on data conservation
and distribution
• The Digital Repositories have to be very useful for
scientists to convince the community (and evaluation
committees)
Scientists (2)
• In astronomy, the usefulness of scientific repositories has been known from a long time
• IUE (1978-1996): five times more publications from data retrieved in the archive than from the selected observing teams
• HST: three times more publications and citations
• A significant and growing fraction of science is multi-wavelength astronomy, hence the VObs – Interoperability increases significantly the impact of the repositorieswavelength astronomy, hence the VObs – Interoperability increases significantly the impact of the repositories
• The data services and tools are part of everyday life of scientists (160 000 queries/day on CDS services in2007)
• But we have to continue to continuously convince the community
• And thus be SCIENCE DRIVEN ( using new technologies when useful) is an absolute requirement
Scientists (3)
• Feedback from scientists
– They do not want to know about the standards and the VObs implementation has to be transparent
– They need a proper description of data
– Most want science ready data (especially for data they are not specialists about)specialists about)
– All want good information about provenance (instrument, observation conditions, data processing)
• A huge work of data specialists to feed the repositories whith data which will be useful and confidently reused
• Discipline specific interoperability standards defined with science usage in mind
Data centres (1)
• Heterogeneous, distributed data
• Oviously, archives of observational data
(which are public after a one year proprietary
period)period)
• But also added-value databases, bibligraphic
data (NASA bibliographic database, electronic
journals), simulation data, …
Data centres (2)
• A huge diversity in size, objectives and context
from international agencies providing archives of their
large telescopes to small teams in scientific laboratories
willing to provide a service focussed on solving a specific
question (‘sharing of knowledge’)question (‘sharing of knowledge’)
• Census of European data centres (EuroVO-DCA)
– More than 70 answers
– Many international connections (data, services)
Data Centres (3)
• Have to listen to the community needs
• QUALITY is a must, and on the long term routine is the worst enemy
• Adaptable, agile strategy: astronomy, and the technical and political contexts change, sometimes technical and political contexts change, sometimes very quickly. Integrate science needs and IT progress.
• ‘Data’ is better curated by people who know about it (observatory archives for data, data centres for value-added services)
Virtual Observatory projects (1)
• Interoperability standards
– A bottom-up approach
– The definition of the ‘proper’ (‘usable’ and ‘useful’)
standards is a huge task, which works when the proper
people from the community are mobilized (archive people from the community are mobilized (archive
providers, data centre staff) – ‘enough’ complexity to start
with, not too much to remain useable by implementers
and understandable by astronomers: a delicate balance
– Take into account feedback from implementation by data
centres and usage by scientists
Virtual Observatory projects (2)
• International cooperation/agreement on interoperability standards is a must
• European coordination/cooperation has been and will remain critical – improves global and will remain critical – improves global results and increases global impact
• R&D/evolution of standards must continue (feedback, scientific and technical evolution, progressive inclusion of more complexity)
• VO-TECH Design Study
(Astrogrid UK, CNRS, ESO, INAF)
• ESA-VO
• EuroVO-DCA
• Architecture (interoperability)
• Tools
• Intelligent rresource discovery
• Data mining and visualisation
• Connection with Grid• Connection with Grid
• Interoperability standards
• Web 2.0, Semantics, data mining
Funding bodies (1)
• Optimize the return of large projects by facilitating the usage and reuse of data
• VObs is a strong tool for world-wide community integration
• OECD recommendation (2004): : “Agencies should encourage broadening of existing VO[bs] collaborations into a fully representative global activity”
• Astronet (ERA-NET) European strategy exercise recognizes the • Astronet (ERA-NET) European strategy exercise recognizes the importance of the VObs and will make recommendations to EC and to the national agencies
• Part of research infrastructure
• New projects MUST include provision for data pipelines, data archiving, data distribution & VObs compliance (and it seems that they understand it)
Funding bodies (2)
• Funding of data centres has to be consolidated in
many cases: European and national agencies,
support from local authorities
• VObs: support needed at different levels for the
operational phase
– Support to national projects
– IVOA is a best effort alliance, medium term commitment of
the national projects needed
– European coordination: how to get support on the
medium term?