information system report

Information System Report-progresses in CSR, EDIOS, EDMED, EDMERP, EDMO, CDI

E. Balopoulos (HCMR), F. Nast (BSH), L. Rickards (BODC), D. Schaap (Maris)

presented by S. Iona (HCMR)

Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece

Outlines

Objectives

Structure

Maintenance

Population tools and updating mechanisms

Current status and system upgrade

Summary


Objectives

The SeaDataNet Information System is an integrated catalogue service, aiming at facilitating marine data searching, location and retrieval.

The Objectives are:

Maintain and expand the national metadata-bases. Standardize the information using common vocabularies

and reference tables. Interconnect the national inventories in a common Pan-

European directory.


Structure

It is composed of several thematic inventories of different levels:

European Directory of Marine Environmental Datasets (EDMED)

Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing

Systems (EDIOS) European Directory of Marine Environmental Research

Projects (EDMERP) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)


Structure


Structure

High Level Directories EDMED, CSR, EDIOSfor describing collected observation datasets (by ships,

by laboratories, by continuous observing system)


Structure

Common Reference Tables EDMERP, EDMOhold research projects and organizations metadata

common to higher directories


Structure

The Common Data Index (CDI)provides access to the data, information and products by data type and/or any other field

distributed by the TA platforms


Maintenance

During the first development phase of the Project (V0) the inventories are organized in centralized databases at the coordinating centres (BSH, BODC, MARIS) based on the scheme of the past SEASEARCH Project:

The TA data centres populate the metadata-bases by compiling, formatting, standardizing their national data and transmitting them to the coordinating centres.

The metadata are accessed through the common metadata-bases that are published as part of the SeaDataNet discovery service: http://www.seadatanet.org/services/catalogues

At V0 the catalogue page links to the SEASEARCH sites.



Through on-line (like CMS forms) and off-line tools that produce XML ISO 19115 compliant exchanges and developed by the Technical Task Team in the joint research activities JRA1,JRA2.

Sea-Search siteLocal interface orJava tool MikadoJava tool MikadoCDI

EDMERP siteAccess DatabaseEDMERP [email protected]

EDMO siteOn lineEDMO

EDIOS siteunder developmentEDIOS

[email protected] ManualLocal interface or MikadoJava tool Mikado

Access DatabaseEDMED

[email protected] site

Local interface orJava tool Mikado

Java tool Mikado(XML converter)On line CSR

Contact and AssistanceDocumentationautomatic XML entriesAlternate

manual entryManual entriesInventory

Need further help? Please contact HCMR/HNODC



NEMO a new reformatting tool to convert ascii files to Medatlas and ODV format (two of the SeaDataNet data exchange formats).

NEMO interacts with Mikado to produce ISO 19115 XML metadata descriptions.

Tools for education and capacity building developed in the frame of the SeaDataNet European project by M.Fichaut et al., IMDIS-2008 Conference.




Cruise Summary Reports (CSR)

European Directory of the initial Ocean-observing Systems (EDIOS)

European Directory of Marine Environmental Research Projects (EDMERP)

European Directory of Marine Organizations (EDMO)

Common Data Index (CDI)


European Directory of Marine Environmental Data(EDMED)

Currently, EDMED describes:

Over 3500 datasets 700 data holding centres

In 2007/2008, BODC received updates from 13 partners, and added 190 new datasets to the database.


Updated EDMEDs since 07/03/2007


Total No. of Datasets and Data Holding Centres


EDMED improvements

BODC are currently developing the software tools that will be used to ingest an EDMED XML record from a partner, containing the metadata from one dataset, into the Master EDMED Oracle schema, where it will be available for retrieval and viewing. Work is also being carried out for the future ingestion of the existing EDMED records.

Software Tools

to carry out the quality control checks on the EDMED record

to ingest the record into the Master EDMED Oracle schema

to allow a partner to retrieve an EDMED record through a web interface


XML validator tool/Quality Assurance

BODC, along with RNODC, have been working to:

Provide an XML validator tool accessible to all SeaDataNet partners via a web browser.

Enable SeaDataNet partners to validate EDMED, CDI, CSR, EDIOS and EDMERP XML documents before they are delivered to the directory object manager for processing.

Enable the object managers to re-validate the XML files before loading the records into the directory database.

Provide a controlled approach to the publishing of XML schemas, and any subsequent versions, which are used to validate the XML documents.

Ensure previous versions of the XML schema files are maintained.


Cruise Summary Reports Directory (CSR)

Number of entries (in an Oracle DB):39 648

New entries since Trieste, 19/03/2007:1263

Characteristics:From 1873-2008, 2105 ships from 48 countries


Continuing operation and populationContinuing operation and population

Heraklion/Triest we got 1071 CSRs

Online 33 % XML 34 % ICES 25 % Others 8 %

Since Triest we got 1263 CSRs

Online 52 % XML 44 % ICES 4 % Others 0 %




2007: 19 countries online submissions

19/36 = 52.78 %

Now: 29 countries online submissions

29/36 = 80,56 %

Entries since last annual meeting Entries since last annual meeting (23.03.2007 (23.03.2007 -- 31.03.2008)31.03.2008)

a big upgrade through France through XML submission, but also by Italy, Tunesia,Turkey, Ukraine

Country Online Totalupdates new updates new

Albania 1 1Algeria 7 7Belgium 1 22 23Bulgaria 39 39Croatia 21 21Cyprus 2 2Denmark 3 2 5Estonia 8 8Faroe Islands 18 18Finland 4 4France 20 247 1 268Georgia 0Germany 125 1 126Greece 29 29Iceland 1 1Ireland 54 54Israel 2 2Italy 12 12Latvia 10 10Lebanon 7 7Lithuania 3 3Malta 3 3Morocco 1 1Netherlands 3 2 5Norway 59 59Poland 9 9Portugal 3 3Romania 12 12Russia 28 28Slovenia 13 13Spain 46 46Sweden 94 94Tunisia 91 91Turkey 38 38Ukraine 14 146 160United Kingdom 20 20

Total 591 20 555 0 48 1214

in percent 49% 4%

XML ICES

CSR Entries since last annual meeting (23.03.2007 - 31.03.2008)

47%


NEW V1 XML FORMAT AND SCHEMA Laboratory, scientist, platform, project now defined by

common vocabularies or by SDN inventories, codes must be used now by CSR V1.2

ISSUE Not all existing CSRs could be harmonised to the

common vocabsACTION-SUGGESTION Partners get export to update those CSRs either online

or by Mikado (V1.2)

Cruise Summary Reportsts Directory (CSR)


EDIOS Progress (1)

No new input has been requested over the year Awaiting new technological developments (XML

schema and new version of Mikado) BODC has produced an improved Oracle

(database) schema Supported by common vocabularies, EDMO and

EDMERP A major issue is population of revised schema with

legacy content and quality assurance against original MIF stock


EDIOS Progress (2)

Within the UK BODC has a national project, UK Directory of the Marine

Observing System (UKDMOS) Adopted the EDIOS schema with some additions Preparing a new UK EDIOS collection

In cooperation with MARIS BODC has defined and prepared an exchange format of

6 csv files Exchange format can work both ways between BODC

database and interface database at Maris Provides basic model for tuning the XML schema later


EDIOS Progress (3)

New User Interface for both UKDMOS and for SeaDataNet - EDIOS

For UKDMOS see:http://www.ukdmos.org/welcome.asp

For SeaDataNet EDIOS:http://seadatanet.maris2.nl/v_edios/search.asp


EDIOS way forward

Two update routes:(1) Exchange format => XML schema => MIKADO version

=> XML output to BODC for validation => export to MARIS for loading in Interface

(2) MARIS builds CMS on the new database model Request partners to rebuild their EDIOS entries Bring in some data from the older EDIOS as basis plus

view on old EDIOS details for comparison when filling in all the Vocabularies

Exchange format in CSV (later as XML) to BODC for loading and validating

Return to MARIS for loading in Public Interface


Total No. of EDMERPs

Total No. of EDMERPs

Albania 0 2Algeria 3 4Belgium 209 128Bulgaria 32 35Croatia 9 7Cyprus 20 9Denmark 40 41Estonia 11 9Finland 16 14France 93 90Georgia 86 102Germany 18 23Greece 114 117Iceland 16 13Ireland 179 160Israel 12 4Italy (OGS +ENEA) 52 48Latvia 31 29Lebanon 5 12Lithuania 28 21Malta 31 30Morocco 5 4Netherlands 71 67Norway 3 4Poland 42 35Portugal 26 26Romania 18 16Russia (SIO + RIHMI) 84 103Slovenia 0 4Spain 91 79Sweden 30 15Tunesia 0 3Turkey 24 28Ukraine 62 79United Kingdom 46 236Other countriesTOTAL 1507 1600

Country

After 1st year SeaDataNet After 2nd year SeaDataNet

EDMERPEDMERP

Number of EDMERP entries is quite stable

Partners have been maintaining existing entries and adding new entries


EDMERD developments

Recently the EDMERP system has been upgraded to Version 1 :

new V1 data format and XML schema, it is fully supported by the Common Vocabularies, double entries have been taken out, because projects are entered

and maintained by the country of the coordinator, who can add via EDMO entries all related partners,

Maintenance is now done by online Content Management System (CMS)and no longer by MS-Access software: capability of creation of sub-accounts for institutes in the NODCs

country, while the NODC safeguards the quality by having the chief editor role before publishing,

New Web service for retrieving EDMERP entries in XML.


Recommendations

Partners are asked to complete the existing EDMERP records with keywords from the Common Vocabularies and partners from EDMO to support the content upgrading from Version 0 to Version 1

Partners are asked to keep EDMERP up-to-date, also with entries from other institutes in their country, because EDMERP is being used in other SeaDataNet Directories


New EDMERP V1 user interface

Browse list

Additional details


EDMERP Content Management System (CMS)


EDMOEDMO

Number of EDMO entries is quite stable

Partners have been maintaining existing entries and adding profile texts, logos and geographic positions

As part of producing CDI records a lot of new organisations have been added; also existing entries have been altered due to re-organisations, double entries etc.

No. of EDMO entries

No. of EDMO entries

Albania - PUT 0 1Algeria 1 1Belgium 63 69Bulgaria 14 12Croatia 5 5Cyprus 3 3Denmark 36 31Estonia 5 5Finland 9 10France 197 198Georgia 13 14Germany 58 58Greece 59 57Iceland 3 3Ireland 104 46Israel 3 2Italy (OGS +ENEA) 66 60Latvia 9 6Lebanon 2 2Lithuania 7 6Malta 7 7Morocco 3 3Netherlands 11 22Norway 7 23Poland 39 40Portugal 43 45Romania 9 8Russia (SIO + RIHMI) 94 88Slovenia 1 1Spain 47 68Sweden 29 32Tunesia 1 1Turkey 25 26Ukraine 12 10United Kingdom 136 153Other countries 17TOTAL 1122 1134

Country After 1st year SeaDataNet After 2nd year SeaDataNet


EDMO recommendation

Recommendation: Try to keep the info up-to-date because EDMO is

being used in all other SeaDataNet Directories (EDMED, EDMERP, CDI, CSR and EDIOS)

Still a number of entries have no logo and no geographic position => please add

Do not keep entries inactive (red button in Content Management System)

In case of new entries, please inform MARIS for making the initial entry


EDMO developments

Upgraded user interface

New option to keep track of no longer existing organisations with a link to present organisations

Google maps in the Content Management System (CMS) for improved placement of geographic locations

Web service for retrieving EDMO entries in XML


Upgraded EDMO user interface


EDMO CMS

EDMO CMS geo-locator via Google maps



European Directory of Marine Environmental Datasets (EDMED) Cruise Summary Reports (CSR)European Directory of the initial Ocean-observing Systems (EDIOS)European Directory of Marine Environmental Research Projects (EDMERP)European Directory of Marine Organizations (EDMO) Common Data Index (CDI)


CDICDI

All partners have been busy and the training workshop has been very useful for transferring expertise and instructing partners in use of CDI format and tools

A number of partners still have no entries, but are busy with producing test entries or production entries on short term. These include:o Albania PUT : has problems ; needs extra instructionso Belgium MUMM: waits for the CDI V1 versiono Estonia MSI: test files almost OKo Iceland MRI: some misunderstanding of CDI dataset definitiono Malta IOI-MOC: test files supplied, but some corrections required o Morocco INRH: test files supplied, but some corrections required o Poland IMGW: first test files supplied; almost ready for production o Portugal IHPT: test files supplied, but some corrections required

No. of CDI entries No. of CDI entries Albania - PUT 0 0Algeria ISMAL 0 17Belgium MUMM 21 21Belgium VLIZ 0 912Bulgaria IO-BAS 0 606Croatia IORS 0 7111Cyprus CYODC 0 1056Denmark NERI 13676 17042Estonia MSI 0 0Finland FIMR 0 1667France IFREMER 71314 74762France - CLS 0 8Georgia TSU 0 99Germany BSH 1594 29295Greece NCMR 10588 11588Iceland MRI 0 0Ireland MI 0 4727Israel IOLR 0 4817Italy - OGS 37926 37926Italy ENEA 0 5395Latvia IAE-UL 0 104Lebanon NCRS 0 7Lithuania CMR 0 810Malta IOI-MOC 0 0Morocco INRH 0 0Netherlands - NODC 5270 41588Norway IMR 13546 14099Poland IMGW 0 0Portugal IHPT 0 0Portugal - IST 0 396Romania NIMRD 0 2461Russia RIHMI 2668 12587Slovenia MBS-NIB 0 2709Spain IEO 0 15547Sweden SMHI 0 2600Tunesia INSTM 0 1497Turkey METU IMS 0 2477Ukraine MHI 0 7476United Kingdom BODC 40122 40122

TOTAL 195131 341499

Country After 1st year SeaDataNet After 2nd year SeaDataNet


Recommendations

Try to keep the info up-to-date, especially the URLs for data access

Missing partners to undertake extra efforts

All partners to prepare for transgression from CDI V0 to CDI V1


CDI developments

Small updates to the CDI V0 user interface

Focus on V1 developments

Synthesis of the technical activity State of development of the on-line system V1by D.Schaap


CDI V0 User Interface

CDI V0 Search results


CDI V0 result


CDI V0 Overview map of point measurements


CDI V0 Overview map of point measurements around Greece


CDI V0 Overview map of point measurements in Black Sea region


Conclusions

Contents of the SeaDataNet Information System:

2006/2007 EDMED: 3.000 CSR: 38.525 EDMERP: 1.507 EDMO: 1.122 CDI: 195.131

2007/2008 EDMED: 3.500 CSR: 39.648 EDMERP: 1.600 EDMO: 1.134 CDI: 341.499

A very important increase in the CDI contents but full coverage of their national data

information system report

Documents