ess vision 2020 admin: assessing quality · wp7. methodological support wp4. eurostat as user of ec...

Eurostat Eurostat

ESS Vision 2020 ADMIN: Assessing Quality

Fabian BACH

ADMIN project manager

Eurostat Unit F.2 – Population and migration

UNECE-Eurostat Expert Meeting on Censuses

Geneva, 27 September 2018

ADMIN data in multisource statistics

Lin

kag

e

Sta

tistic

al

reg

iste

r

Outputs

Po

pu

latio

n

Reg

iste

r

Tran

sfo

rm

atio

n

`

use for survey frames: directly or to supplement / update

(partly) replace data collection: e.g. business tax data survey

direct tabulation

Graph idea: ONS

Registers

Admin. data

Surveys

Full enumeration

ADMIN data in multisource statistics

Lin

kag

e

Sta

tistic

al

reg

iste

r

Outputs

Po

pu

latio

n

Reg

iste

r

Tran

sfo

rm

atio

n

`

Graph idea: ONS

Registers

Admin. data

Surveys

Full enumeration

(mostly) national level! (ESS) quality??

Inputs

Key ADMIN challenge

• as a single variable

• editing, imputation, validation

• indirect use

(calibration, estimation, …)

• in evaluation, including data

confrontation

(e.g. comparison of survey

estimates with related ADMIN

estimates)

Generic Statistical

Business Process Model =

Multisource data: use examples

ESS Vision 2020 ADMIN project (2015-2019)

WP2. Statistical methods

WP5. Frames for social statistics

WP6. Pilot studies &

applications

WP7. Methodological

support

WP4. Eurostat as user of EC data

WP1. Access to & development

of sources

Approved by ESSC: support the ESS to make better use of available ADMIN data without compromising on quality

Work

Packages (

WP)

WP3. Quality

ADMIN: key results

Multisource combination +

estimation methods

Access to ADMIN sources

Use of registers for frames

2019 Q2/2018

Quality of frames for social statistics today!

Multisource quality

Guidelines

WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer

Best practices

Incl. 2017, 2018 waves

ADMIN: key results

Multisource combination +

estimation methods

Access to ADMIN sources

Use of registers for frames

Quality of frames for social statistics today!

Multisource quality

Guidelines

WP6 grant summary reports social/business/agricultural stats. + quality lessons learned + knowledge transfer

Best practices

Incl. 2017, 2018 waves

Stress on social/pop./ census statistics through all ADMIN!

Reports to ESS Directors

(methodology, social statistics)

ADMIN: Involving ESS Members

Quality of multisource statistics (DK, IE, IT, LT, HU, NL, AT, NO)

• Stock-taking • New indicators • Methodological framework quality guidelines (x2)

ADMIN Steering Group

Governance technical

ADMIN network

Task Force Frames

ESSnet KOMUSO

Workshops

ADMIN WP3: Quality (KOMUSO)

Komusō (image: Wikipedia)

ESSnet

Task 3.1 – Checklists for

evaluating the quality of input data

Task 3.2 – Definition of a

framework for the quality evaluation of statistical outputs using administrative data

Task 3.3 – Dissemination and

practical implementation of main results obtained in tasks 3.1 and 3.2 write guidelines!

https://ec.europa.eu/eurostat/cros/content/essnet-quality-multisource-statistics-komuso_en

https://en.wikipedia.org/wiki/Komus%C5%8D










2015 until 04/2017:

Outputs published:

KOMUSO Stage 1: Preparatory Work

Take stock on output quality assessment +

reporting Review output quality measures

Review current practices on input quality assessment

• checklists for evaluating the quality of input data

• framework: 6 basis data source configurations

• quality measures and indicators (for frames)

Review existing practices on frames

quality

1st ADMIN quality workshop

(Budapest, 04/2016)

6 basis data source configurations:

1) Non-overlapping microdata without coverage problems

2) Overlapping microdata without coverage problems + (2S) including sample surveys

3) Overlapping microdata with under-coverage

4) Combining microdata + macrodata sources

5) Combining only macrodata sources

6) Longitudinal data

Selected to reflect the most common/typical scenarios


KOMUSO Stage 2: Guidelines

04/2017 until 09/2018:

Outputs published (continuing stage 1):

Finalise draft frames guidelines

Continue work on measures/indicators

by data config.

1st progress draft of multisource guidelines

• Quality measures and calculation methods (QMCMs)

• Quality quidelines for frames of social stats.: QGFSS v1.0

• Quality guidelines for multisource statistics: QGMSS v0.7

accuracy (18) coherence (3) timeliness (1)

Structure + drafting progress of the QGMSS v0.7:


Structure + drafting progress of the QGMSS v0.7:


…

…

QMCMs (annex)

Same structure for each section in Part 2:

Excursion: structure of QMCMs (annex to QGMSS)

1) Name of the quality measure/indicator

2) General information

3) Quality measure & computation method in general

4) Situation & computation method in detail

5) References

+ hands-on example where needed


E.g. QMCM_A_10 (Accuracy example 10):


… …

1)

2)

3)



… …

(config. 2)

1)

2)

3)


4)

5)



4)

5)


Detailed calculation algorithm/recipe

E.g. QMCM_A_10 + hands-on example:


…

E.g. QMCM_A_10 + hands-on example:


…

11 steps: Raw input data Combinded DB Final data pool

Multisource quality propagation!

KOMUSO Stage 3: Completion

10/2018 until 09/2019:

Outputs envisaged (continuing stages 1+2):

Finalise measures/indicators QMCMs by data config.

Finalise draft multisource

guidelines (QGMSS)

accuracy contd. (+6) coherence contd. reliability (1) + hands-on examples

• QMCMs contd. incl. hands-on examples

• QGMSS v1.0 tentatively June 2019

2nd ADMIN quality workshop

(winter '18 tbc.)

ADMIN support to Member States

• Workshops: quality + social statistics + other

• (WP6: Grants for ADMIN pilots in Member States)

• WP7: Methodological support

ADMIN helpdesk for NSIs http://ec.europa.eu/eurostat/cros/content/ess-admin-helpdesk_en

Coaching for NSIs (on-site expert support) request through ADMIN helpdesk

ESTP training linked to ADMIN in 2017-2019

winter 2018 tbc.

spring 2019 tbd.

http://ec.europa.eu/eurostat/cros/content/ess-admin-helpdesk_en





Thank you!

CROS portal: http://www.cros-portal.eu/content/essvip-admin

[email protected]

http://www.cros-portal.eu/content/essvip-admin









mailto:[email protected]





Annex:

6 basis data source configurations

1) Non-overlapping microdata without coverage problems:


most simple

still typical errors:

classification

measurement

progressiveness

incl. single source

reference config.


2) Overlapping microdata without coverage problems:

configuration (1) +

linkage errors

inconsistencies

most common configuration


2S) Overlapping microdata without coverage problems:

incl. samples

configuration (2) +

sampling errors

weights handling

calibration


3) Overlapping microdata with under-coverage:

configuration (2) +

under-coverage

may include samples ( 3S)


4) Combining microdata + macrodata sources:

configuration (2) +

reconciliation of microdata (calibr.)


5) Combining only macrodata sources:

macro-counterpart of configuration (2)

reconcile overlapping data

E.g. supply/use (SU) tables of National Accounts


6) Longitudinal data:

reconcile time series

different frequencies and qualities

now-casting (time series model)

linkage errors

ess vision 2020 admin: assessing quality · wp7. methodological support wp4. eurostat as user of ec...

Documents