data management and qa/qc with priority data flows in eea

1

Data management and QA/QC with Data management and QA/QC with priority data flows in EEApriority data flows in EEA

88thth Joint TFEIP/EIONET meeting Joint TFEIP/EIONET meeting

Dublin, 23-24 October 2007Dublin, 23-24 October 2007Sheila Cryan, EEASheila Cryan, EEA

2

Priority data flowsPriority data flows

• EEA and EIONET set of agreed, stable, well-defined objectives

• provide a focus for countries when they are putting procedures in place for regular reporting.

• better performance through “compétition amicale”

• praise for achievements rather than blame for failures.

3

Who and what?Who and what?

http://www.eionet.europa.eu/dataflows

4

Regular annual cycleRegular annual cycle

5

Data reporting in 1990’s… Data reporting in 1990’s…

The Public and Decision-Makers

NFP, other National AuthoritiesPCP, NRC

EEA

Bigreport

consultants

ETCACC

ETCBD

ETCWater

CEC Others

6

What is Reportnet? What is Reportnet?

Reportnet is a system of integrated IT tools and processes creating a shared information infrastructure to support the reporting of environmental dataand information in Europe

7

EEA

The Public and Decision-Makers

ETCACC

ETCBD

ETCWater

NFP and other National AuthoritiesPCP, NRC

ESTAT DG ENV

Others

Reportnet……and after :-)and after :-)

8

EE

A

NR

C N

RC

NR

C N

RC

NF

P

ET

C E

TC

DataCollection

DataHandling

Disseminationof information

9

Rotating ReportnetRotating Reportnet

10

Reportnet Reporting Obligations Database

Reporting obligations [Acidification]

11

Definitions in Reportnet Data Dictionary

12

... > Tables > Elements > Code lists

13

Reportnet Central Data Repository

14

Immediate feedback to the data reporter

15

2nd level data quality checks

outlier

median

max. tolerated value = median * 1.5

min. tolerated value = median / 2

no outlier

outlier?

example chart

outlier

outlier

outlier

outlier

16

Reportnet, version 1bReportnet, version 1b

Contentmonitoring

Informationproducts

EEAdata

services

Reportingobligations

Networkdirectory

Datadictionary

Dataexchangemodules

Datarepository

17

QA/QC – Summary

1. National level: MS responsibility

2. EU level: Data quality index

3. EU level: Validation of reported data The Reportnet system is designed to support

quality control of reported data. A clear specification of data elements, data

types and code lists is needed. Data element specification in most reporting

sheets need further elaboration so that meaningful quality checks can be carried out.

18

We do not have a magicdata washing machine!

The good thing about data quality is that literally anyone who wants

can contribute to its improvement.

19

20

Content in ReportnetContent in Reportnet

3716 envelopes 3716 envelopes

(8 October 2007)(8 October 2007)

21

Informationproducts

Contentmonitoring

Datadictionary

Reportingobligations

EEAdata

services

Dataexchangemodules

Networkdirectory

Datarepository

Data handling

Data reporting Data dissemination

Reportnet under the magnifying glassReportnet under the magnifying glass

22

Datarepository

New features 2004-2007New features 2004-2007

Contentmonitoring

2004GDEM = Converters + Webforms + QA2005More (automatic) convertersData Merging Module (DMM) = AggregatorUNS = Content monitoring, CAS2006Improved handling of GIS data:i.e. conversion, editing, display and QA2007 (?)

Default workflow >> Customised workflow