high output quality monitoring via the ecb quality reports on statistics pedro abad banco de españa...
TRANSCRIPT
HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS
Pedro Abad
Banco de España
European Conference on Quality in Official Statistics
Viena, 2-5/06/2014
STATISTICS DEPARTMENT 2
HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS
1. Introduction. Main elements of the Statistics Quality Framework: towards high-quality output
2. Quality monitoring and reporting on financial
accounts
Quantitative quality indicators
3. Output enhancement from the ECB quality monitoring reports: the experience of Banco de España
4. Conclusions
STATISTICS DEPARTMENT 3
1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT
The ECB statistical system rests on three pillars:
the regulations setting obligations on economic agents
the guidelines given to NCBs detailing the datasets to be transmitted and other statistical requirements, and
the quality framework
The ESCB Public Commitment on European Statistics set the principles that govern the collection, compilation and dissemination of statistics by the ECB.
The ECB Statistics Quality Framework combines these principles and others of good governance with the quality assurance procedure developed by the ECB.
Quality reports are at the core of the system as they review main quality elements and draw some conclusions/recommendations in order to enhance statistics.
STATISTICS DEPARTMENT 4
1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT
Quality is defined in relation to how well statistics are fit for purpose, and, therefore, meet users´ and other stakeholders´ expectations. This is a multidimensional concept.
Guiding principles from the Public Commitment on European Statistics by the ESCB
Institutional environment Statistical processes High output quality
1 Professional independence 7 Sound methodology 11 Relevance
2 Mandate for data collection 8 Appropriate statistical procedures
12 Accuracy and reliability (including stability)
3 Adequacy of resources 9 Minimisation of the reporting burden
13 Timeliness (including punctuality)
4 Commitment to quality 10 Cost-effectiveness14 Consistency and comparability
5 Statistical confidentiality 15 Accessibility and clarity
6 Impartiality and objectivity
STATISTICS DEPARTMENT 5
1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT
Data Quality Framework: High Output Quality
Principles Quality assurance procedures Principles Quality assurance procedures
Relevance
Medium-term strategy / annual work programmes
Merit and cost exercises Seminars and consultations
with users
Consistency and comparability
Reporting schemes follow international accounting standards and best practices
Regular internal quality monitoring reports
Accuracy and reliability (including stability)
Reporting schemes follow international accounting standards and best practices
Completeness and plausibility checks
Revision analysis and consistency analysis between different datasets
Regular internal quality monitoring reports
Accessibility and clarity
Intensive use of up-to-date electronic system for disseminating data
Press releases, regular publications
Data warehouse, joint country-tables
International standards in the denomination of variables
On-line documentation
Timeliness (including punctuality)
Transmission calendars Release calendars Compliance with IMF SDDS
standards
STATISTICS DEPARTMENT 6
2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS – QUALITY REPORTS
The main two procedures implemented by the SQF to report on quality to the ECB senior
management are: The audit reports prepared by the Internal Auditors Committee (IAC) Quality reports prepared by the Statistics Committee (STC)
Quality reports prepared by the STC review main quality elements with the help of some
usually pre-defined indicators and draw some conclusions and/or recommendations in order to
enhance the quality of statistics
Several ECB statistical guidelines require the preparation of quality reports, mainly focused on
the compilation of euro aggregates: Balance of Payments and International Investment Position Statistics (biannual), Government Finance Statistics (internal), Monetary and Financial Statistics, Centralised Securities Database (internal) Financial Accounts.
All these reports are based on a common framework, although they are tailored to each
specific statistical domain and so their content is not fully homogenous.
The preparation of these reports by the ECB with the assistance of the expert groups can be
seen as the final step of the quality monitoring process, which also includes other internal
quality reports.
STATISTICS DEPARTMENT 7
2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS
ECB compiles euro area financial accounts on the basis of:
national financial accounts transmitted under Guideline ECB/2013/24, and
datasets already available in the ECB data warehouse from other ECB
guidelines. The most important are: banking statistics, securities statistics,
government finance, and external statistics.
ECB Guideline requires the transmission of:
stocks, transactions, other flows and counterpart information for selected items
information on major events and revisions
the preparation of an annual quality report including at least: coverage,
compliance with relevant definitions and revisions
The quality monitoring process carried out by the ECB with the assistance of the expert group on financial accounts also includes several internal quality reports.
STATISTICS DEPARTMENT 8
2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS
Table 2. Coverage of quality monitoring reports on financial accounts
Periodicity OrientationProduction
Collection Compilation Dissemination
Internal
quality monitoring reports
Feedback country report Quarterly Compilers X
Regular transmission report (EA level, information by country)
Quarterly Compilers X X
Integration report (EA level) Quarterly Compilers/
internal users X
Methodological notes (EA level) Ad-hoc Compilers/
internal users X
Quality report(EA level, public) Annual Internal users/
public X X X
STATISTICS DEPARTMENT 9
2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS
It is inspired by the basic principles of the SQF Public (since 2013) Coverage:
Institutional and legal framework Quality assurance procedures and user involvement Coverage Revision analysis Consistency and comparability Timeliness of national contributions Data accessibility Recent developments NCB-specific assessment, recommendations and non-
compliance cases (only internal version)Comprises a set of quantitative indicators at EA level
EURO AREA QUARTERLY FINANCIAL ACCOUNTS REPORT
STATISTICS DEPARTMENT 10
2.1 QUANTITATIVE QUALITY INDICATORS
Revision analysis
Errors or revisions are defined as the difference between the current values of a variable (as transactions) and the values when they were released for the first time. The quality report presents four indicators to assess accuracy and reliability.
Positive revisions (R) as % of positive revisions in relation to total observations
Directional reliability (Q) as % of first release of data with the same sign as the last release
MACE [Mean Absolute Comparable Errors] which is the mean of absolute errors on transactions in relation to stocks, and
RMSRE [Root Mean Square Relative Errors] which is a relative measure of errors in relation to the variability series
Consistency analysis
Statistical discrepancies arise in the compilation process due to the integration of different data sources. The quality report assesses discrepancies in financial instruments (horizontal inconsistencies), and between the capital and the financial accounts (vertical inconsistencies) by means of the RMSE indicator.
RMSE [Root Mean Square Errors]
STATISTICS DEPARTMENT 11
2.1 QUANTITATIVE QUALITY INDICATORS
Euro area quality reliability indicators
for households
STATISTICS DEPARTMENT 12
3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA
The BdE contribution is fully compliant with the ECB data requirements
The BdE takes note of the conclusions and recommendations to enhance, as far as possible, the Spanish financial accounts and therefore the national contributions to the ECB
In recent years, changes have been introduced in several areas following quality monitoring reports: to reduce inconsistencies with securities issues statistics due to the
treatment of own securities used as collateral at the ECB to reduce inconsistencies with money and banking statistics due to
the treatment of factoring operations related to general government transactions
to harmonise the treatment of repos transactions across euro area countries
other discrepancies between data sources are regularly monitored Discrepancies between net lending/borrowing and net financial
transactions of non-financial corporations and households are monitored by the BdE and the INE
STATISTICS DEPARTMENT 13
3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA
Spain. Discrepancies between net lending/borrowing and net financial
transactions
STATISTICS DEPARTMENT 14
4. CONCLUSIONS
Monitoring and reporting on quality is an essential element of the Statistics Quality Framework.
Several ECB Guidelines require the preparation of annual quality reports, which to some extent are the final step of the quality monitoring process carried out by the ECB with the assistance of the expert groups.
By means of ECB quality reports the Statistics Committee can draw the attention of the highest decision-making bodies of the ECB to relevant quality problems on euro area statistics, and related national contributions, so that the ECB and national central banks can resolve underlying problems.
The quality report on financial accounts is an example of this strategy. This and other internal quality monitoring reports cover the compliance by countries with the ECB regulations and other quality aspects, in particular revisions and consistency with other statistics, which are examined on the basis of a set of quantitative indicators.
Finally, the paper provides some information about the changes introduced in the Spanish financial accounts following the conclusions and recommendations of the ECB quality reports.
STATISTICS DEPARTMENT
THANK YOU FOR YOUR ATTENTIONPedro Abad