simorc workshop march 2007 seadatanet a pan-european infrastructure for ocean and marine data...

SIMORC Workshop March 2007

SeaDataNet

A Pan-European Infrastructure for Ocean and Marine Data

Management


Background

• Data availability of vital importance for marine research

• European data set is fragmented

• Not always quality controlled

• Not always easily accessible

• In 35 countries bordering European seas• more than 600 scientific laboratories from government

organizations and private industry collect data

• use various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites

• measure physical, geophysical, geological, biological and chemical parameters, biological species, etc.


What is SeaDataNet?

• EU Sixth Framework Programme • Integrated Infrastructure Initiative (I3)• 1 April 2006 to 31 March 2011• Objectives

• develop an efficient distributed pan-European marine data management Infrastructure for managing large and diverse data sets

• network existing professional data centres of 35 countries, active in data collection

• provide integrated databases of standardised quality on-line


SeaDataNet partners

49 partners including:• Major oceanographic

institutes of 35 countries

• National Oceanographic Data Centres (NODCs)

• Divisions of major national marine research institutes

• 3 satellite data centres

• 3 scientific modelling centres

• 2 SMEs experts in software development

• 3 International organisations


Building on experience

• Data management of scientific projects, e.g. MTPII-MATER, CANIGO, OMEX, MERSEA, etc.

• Leaders of Data Management Concerted Actions• MEDAR/MEDATLAS II - Mediterranean Data Archaeology

and Rescue, aiming to make available a comprehensive data product of temperature, salinity and bio-chemical data in the Mediterranean and Black Sea through a wide international cooperation

• SEA-SEARCH – The gateway to Oceanographic and Marine Data & Information in Europe, to , to provide an effective navigation tool to oceanographic and marine data and information sources for European marine waters and to centres in Europe with expertise and skills in oceanographic and marine data & information management.

• Bringing in new partners – satellite data centres


SeaDataNet components


Infrastructure, versions & planning

• Version 0Continuation and maintenance of existing Sea-Search systems with minor modifications

• Version 1All data centres involved in upgraded meta-databases

Technical Task Team partners involved in transparent data access, security, monitoring, viewing services = trans-national access

Pre-operational by October 2007 and operational by February 2008

• Version 2All data centres involved in trans-national access and access upgraded

Operational by 2010


SeaDataNet data policy

• Metadata• free and open access, no registration required• each data centre should provide the meta-data in standardized

format to populate the catalogue services

• Data and products services • the general case if free and open access/mandatory registration• a “SeaDataNet role” (partner, academic, commercial etc.) is

attributed to each user • Each NODC attributes the roles to the users of its of country• Out of the partnership, the roles are assigned by SeaDataNet user-

desk

• each data centre node delivers data according to the user’s role and its local regulation

• each data centre should provide freely the data sets necessary to develop the common products


Discovery services

• Marine environmental data sets (EDMED)

• Marine environmental research projects (EDMERP)

• Observing systems (EDIOS)• Research Cruise Summary

Reports (CSR) • Common Data Index (CDI)• Directory of marine

organisations (EDMO)


Cruise Summary Reports

• Conceived by IOC• ICES acted as focal point • Currently over than 37000

cruises• Documenting data

collected on scientific research cruises operated by European vessels

• Cruise track charts included

• Online entry form• Online searchable

database


European Directory of the Ocean-observing System (EDIOS)

• Initiative of the European Global Ocean Observing System (EuroGOOS)

• Regular repeated measurements

• On-line searchable directory of observing, measuring, and monitoring systems

• Over 10,000 observation entries

• Links to on-line real-time and archive data

• Tool to aid design of observing systems


Common Data Index

• Index to individual data sets

• Approx. 200000 entries from 10 partners

• ISO19115 metadata standard has been adopted

• On-line searchable directory

• Direct access to on-line data sets, catalogues or e-mail request


Technical developments

• Establishment of common vocabularies

• Upgrading formats for interrelationships between these directories

• ISO 19115 content model is the basis for the XML formats and exchange schema (XSD)

• Innovating maintenance modalities:• XML exchange formats upgrading

• XML validation services

• Online Content Management System (CMS)

• Introducing Web services• Vocabularies and Directory of Marine Organisations (EDMO)


SeaDataNet standards

• Interoperability – key to distributed data management system success• Includes establishing common understanding of terms

used• Achieved through use of controlled vocabularies

containing keys, terms, abbreviations and essential (but often missing) definitions

• Schemes for handling duplicate data• Common quality control protocols• Ocean Data View software for visualisation


SeaDataNet : Delivery Services V1

Primary function• Users will be able to download

selected data sets CDI User Interface• Searching and browsing of

metadata of data sets

Shopping Basket in combination with AAA services• Preparing a user request of multiple data sets, handling

validation of users, and routing requests to the data centres


Authentication, Authorization and Administration (AAA)

• Single Sign On system required for access to distributed system

• User’s authentication information based on personal login / password

• User must register in order to get login• Web form to provide necessary information • Online user agreement “SeaDataNet General Licence”• After processing, login/password sent by e-mail (e-mail check)

• Users’ directory management (registration processing)• Centralised vs Decentralised

• Choice in V1 for CAS system (= centralised) • SeaDataNet Data Policy • Authorisation based on “Roles”


SeaDataNet : Delivery Services V1

Download manager• process of requests by each data centre to deliver one

comprehensive data set (one zip file) per data centre

Request status manager• provide users overview of request status of requests and

download functions• provide data centres with overview of all requests and

data delivery

Data delivery formats

• CF compliant NetCDF format for gridded and multi-dimensional data types (e.g. ADCP)

• Modified ODV spreadsheet format for other data types (e.g. time-series, profiles)


SeaDataNet User’s portal schema V1

IFREMER Database

BODC Database

BSH Database

...

Download managers in data centres

CSR EDIOS

EDMED

CDI

Shopping basket

Requestsstatus

manager

datametadata

CSR

CDI

Project info

Metadata & Data catalogues

Data request Status of request

General request Metadata request

Organisation +data source id

Data download

at BODCat BSH at MARIS+ EDMERP

services for metadata

Data downloading services

Visualisation services (WMS) for V2

Entry point for access hits

EDMO

EDIOS EDMED

EDMERP EDMO

Software

Vocabularies

Standards Cross search

AAA

Registr.

My transact.

User Register

Userregistration


Data products

• SeaDataNet will provided added-value products to be created by regional centres• Mediterranean Sea• Black Sea• Baltic Sea• Arctic and North Sea• Atlantic and global ocean

• First products => gridded climatologies • based on already available data-sets (mostly hydrographic data)• updated during the project life

• To make products comparable, all regional data centers will use a common tool for spatial mapping

• For this a new version of DIVA (Data Inverse Variational Analysis) was developed (University of Liege)


SeaDataNet capacity building

The training program has two main objectives:

•to ensure that the data and metadata to be integrated in the SeaDataNet system will be formatted, checked for quality and disseminated according to the common protocols developed during the project

•to transfer expertise and to train IT experts of the SeaDataNet data centers in the basics, installation and operation of the SeaDataNet technical components


SeaDataNet system components

• Discovery services = Metadata directories

• Vocabulary services = Common vocabularies

• Auxiliary tables = Organisations, Research Vessels

• Security services = Authentication, Authorization & Administration (AAA)

• Monitoring and statistics = usage & system performance

• Delivery services = Data access & downloading of data sets (ASCII, NetCDF, …, data virtualisation)

• Viewing services = GIS, ODV analysis software, Google Earth, ..

• Product services = Generic and standard products


International Conference IMDIS 20081st Announcement

• Second international conference “International Marine Data Information Systems” (IMDIS)

• March 31 - April 1, 2008, Athens, Greece • Joint organisers: SeaDataNet, Hellenic Centre for

Marine Research, IOC/IODE • Four sessions:

• Marine environmental data bases: infrastructures and data access systems

• Data Services in ocean science: standards and interoperability

• Visualisation tools• Services for Users and Education


www.seadatanet.org

simorc workshop march 2007 seadatanet a pan-european infrastructure for ocean and marine data...

Documents

data collection

marine data management

transparent data access

catalogue services data

biochemical data

mediterranean data archaeology

simorc workshop

data use various sensors