international virtual observatoryopenuniverse.asi.it/documents/presentations/ivoa... · metadata,...

International Virtual Observatory

AllianceThe Standards Organization

for Data Interoperability in Astronomy

G. FabbianoHarvard-Smithsonian Astrophysical Observatory

Chair of the IVOA Executive Committee

Rome, 4/11/17 Pepi Fabbiano - [email protected] 1

What is the Virtual Observatory?

A multi-wavelength digital sky that can be searched,

visualized, and analyzed in new and innovative ways

Space, Ground, and Theory data


What is the Virtual Observatory?The latest stage of good data practices in

astronomy

• Each data center / archive with different file structure,

metadata, table organization

• The data is there, but it takes work to access it, especially if

several data sets are used together



astronomy





Rome, 4/11/17 Pepi Fabbiano - [email protected]

• FITS provided a first standardization

4


astronomy






• FITS provided a first standardization

• The Virtual Observatory is the natural

progression towards interoperability of data,

services and tools

5



Data management & Access

InteroperabilityBig Data

6





7





8





9


Paradigm for supporting interdisciplinary and collaborative

research, education, outreach and citizen science in

astronomy, exploiting the full power of growing and

emerging data sets

• The VO is not a fixed system, but rather a way of doing

things, a framework

– For data centers to provide co-operating data services,

– For software providers to offer a variety of compatible analysis

and visualization tools and user interfaces


IVOA StandardsAt the core of the VO is a set of standards developedwithin the International Virtual Observatory Alliance(IVOA) to support good data management and

interoperability

• Data and metadata for observed and simulated data– Data Model; Uniform Content Descriptors

• Data exchange methods – Data Access Layer; VO Query Language

• Lists and characteristics of available services – IVOA registry

• Application messaging protocol– SAMP

– VO-enabled tools and services can interface seamlessly with VO-enabled archives worldwide


IVOA StandardsAt the core of the VO is a set of standards developedwithin the International Virtual Observatory Alliance(IVOA) to support good data management and

interoperability

• Data and metadata for observed and simulated data– Data Model; Uniform Content Descriptors

• Data exchange methods – Data Access Layer; VO Query Language

• Lists and characteristics of available services – IVOA registry

• Application messaging protocol– SAMP

– VO-enabled tools and services can interface seamlessly with VO-enabled archives worldwide


New data systems with VO at their core are

being developed world-wide

e.g., CADC, Gaia, Euclid,

Australian Sky Mapper Database….

IVOA Todayhttp://www.ivoa.net

• Created in 2002, the International Virtual Observatory Alliance (IVOA) today has 23 member projects– 2 well attended interoperability working meetings per year

• The IVOA has no direct funding– Affiliated projects seek funding from their national agencies

– Members have come and gone, some projects have persisted and some others have stopped

• Hard lessons learnt on management and developing standards– Management framework and roadmaps

– User participation is important

• IVOA has survived because VO is a good idea


http://www.ivoa.net









http://www.ivoa.net





• Hard lessons learnt on managing the development of standards– Management framework and roadmaps




http://www.ivoa.net









http://www.ivoa.net

IVOA Organization Chart


ExecutiveCommittee

Technical Coordination Group

DataModel

Data AccessLayer

Applications

SemanticsRegistryGrid & Web Services

Knowledge Discovery in Databases

OperationsTime Domain

TheoryEducationData Curation& Preservation

Committee on Science Priorities

Standards & Processes Committee

Working Groups Interest Groups

17

IVOA Successes


Stable Architecture Stable Processes

Documents accessible via www.ivoa.net

and NASA ADS (by authors)

18

http://www.ivoa.net

IVOA Successes –Archives in the

VO• Major astronomical data collections accessible through the VO

– CDS, CADC, MAST, IRSA, ESA, Chandra, …


IVOA Successes – VO in Data Centers

• VO-ready infrastructure built-in astronomical data centers

– CADC, Gaia, Euclid, …

– Data (file & database) access and User Work Space (VOSpace)


IVOA Successes – Reliable Operations

• Reliability of VO Services Operations

– Hundreds of VO services in the IVOA

Registries…

• Compliance of VO services ?

• VO Services validators

– IVOA Operations Interest Group monitors

operationsChair: Tom McGlynn, HEASARC (NASA-GSFC)


IVOA Successes – Seamless Science

A new paradigm for research

Integrated archive – literature - analysis sessions, through: • IVOA-compliant user interfaces

• Query protocols

• SAMP – VO interoperable applications

• ADS-archive two–way links (e.g. Chandra)


VO interoperable applications

• Topcat, Aladin, VOSpec, SPLAT-

VO, Iris, DS9 …


Chandra Source Catalog

Thread


VO

Python scripting

VO Data

TopCat

CSCView

DS9 24

Chandra Source Catalog

Thread


IVOA Successes – Education, Outreach &

3rd-party applications


• EuroVo for Education

– Suite of IVOA – compliant tools

and use cases– http://vo-for-

education.oats.inaf.it/eng_download.html

• WorldWide Telescope

(Microsoft)

– VO inspired and VO compliant

– Now under AAS

26

http://vo-for-education.oats.inaf.it/eng_download.html

New Project Engagement in IVOA

• Standards development is not a top-down activity

• Project input and participation are vital

• New science projects are welcome to participate in IVOA standards definition and development


New Project Engagement in IVOA

• The IVOA Committee on Science Priorities seeks input for science drivers – identifies fast-track standards

• Organizes focus session with projects – e.g. Cape Town IVOA meeting, May 2016

“Interoperability of data from major astronomical projects”


Presenter Project: Multi-l, Space and Ground

R. Taylor LSST

M. Allen et al. ASTERICS: E-ELT, SKA, CTA, LOFAR, …

M. Zhu FAST

J. Chapman ASKAP

B. Merin Euclid, Gaia

T. Donaldson JWST

28

The VO is not a Killer Application


Dr. Strange

29


The VO is an Evolving Ecosystem

IVOA Organization

• Review/Approve standards to ensure processes are followed

• Prioritize work with input from CSP and TCG (see below)

• IVOA Chair, IVOA vice-Chair (rotating, 18 months term)

• Representatives from all national VO projects

Executive Committee

• Identify research needs that can benefit from VO related tools and services

• Take action within the context of the IVOA to assist in placing such tools and services into the research community

Committee on Science Priorities

• Develop IVOA interoperability standards to support science cases

• TCG Chair / Vice Chair coordinate activities and maintain roadmap

• 6 Working Groups defining interoperability standards (Applications, Data Access Layer, Data Model, Grid and Web Services, Registry and Semantics)

• Each WG has Chair and Vice Chair

• 6 Interest Groups (Data Curation and Preservation, Education, Theory, Time Domain, Operations, Knowledge Discovery in Databases)

• Standard and Processes Committee

Technical Coordination Group


IVOA Working Groupswhere the standards are developed

• Applications– Interoperability between Astronomy data, services and software empowers

astronomers to combine these for scientific discovery and analysis

• Data Access Layer– Define and formulate VO standards for remote data access

• Client data analysis software use these services to access data via the VO framework

• Data providers implement these services to publish data to the VO

• Data Model– Framework for the description of metadata attached to observed or simulated data

– Logical relationships between metadata and architecture to handle them • How does an astronomer wants to retrieve, process and interpret astronomical data?

• Grid & Web Services– Define the use of Grid technologies and web services within the VO context

– Investigate, specify, and implement required standards

• Registry– Provides the mechanism for users and applications to discover and select resources

• data and services

• Semantics– IVOA Standard Vocabulary

– Exploration of Ontologies
