EUROSTAT

Directorate F: Social statistics

Unit F1: Social indicators; methodology and development;Relations with users

http://ec.europa.eu/eurostat

Flash estimates

of income inequalities and poverty indicators for 2016 (FE 2016)

Experimental results

September 2017

2

1. Providing flash estimates one year earlier

Providing timelier social statistics – especially indicators on income poverty and inequality – is a

priority for the Commission and the European Statistical System.

Indicators on poverty and income inequality are based on EU statistics on income and living

conditions (EU-SILC). These indicators represent an essential tool to monitor progress towards the

Europe 2020 poverty and social exclusion target and to prepare the European Semester (the annual

cycle of economic policy coordination between EU countries).

In 2017, EU-SILC income indicators for 2015 (SILC 2016) will be available for all countries by

autumn, which is late for the EU’s policy agenda. Efforts for improving the timeliness of EU-SILC

data are ongoing but the collection and processing of EU-SILC data based on both survey and

administrative sources, will always have a certain time lag.

In order to better monitor the effectiveness of social policies at EU level, it is important to have

timelier indicators. A new approach was therefore proposed, which consists in the development of

flash estimates (FE). These are calculated on the basis of a statistical or econometric model and have a

release date appreciably earlier than the actual data: i.e. flash estimates of income 2016 (SILC 2017)

are available in September 2017. These will complement the EU-SILC data and can be used in

preliminary discussions and analysis until the final EU-SILC data become available in the summer

2018 for most countries.

2. What are the flash estimates on income distribution?

Flash estimates refer to a set of key income indicators:

a. At-risk-of-poverty1 (AROP) & Income quintile share ratio

2 (QSR) are inequality

indicators, both high on the priority of the Commission, Eurostat and the European Statistical

System (ESS). They are used by policy makers at EU and national level for monitoring of

progress towards the Europe 2020 poverty and social exclusion target, for preparing the

European Semester and for identifying the key social trends.

b. Evolution of income3 deciles (D1, D3, MEDIAN, D7 and D9) can provide useful

information on the developments within different parts of the income distribution. The deciles

can provide support for assessing the yearly changes in the distribution: they are more

sensitive to income changes and therefore can be informative as early warnings as well as for

better explaining the estimated changes in inequality indicators.

1 http://ec.europa.eu/eurostat/statistics-explained/index.php/Glossary:At-risk-of-poverty_rate 2 S80/S20 ratio 3 http://ec.europa.eu/eurostat/statistics-explained/index.php/Glossary:Equivalised_disposable_income

3

Table 1. Definition of the inequality and income distribution indicators

Indicators Definition

At-risk-of-poverty

rate (AROP)

Share of people with an equivalised disposable income4 (after

social transfer) below the at-risk-of-poverty threshold, which is set

at 60 % of the national median equivalised disposable income

after social transfers.

This indicator shows the percentage of the population whose

income is likely to "preclude them from having a standard of

living considered acceptable in the society in which they live"5.

Income quintile

share ratio (QSR)

The ratio of total income received by the 20 % of the population

with the highest income (the top quintile) to that received by the

20 % of the population with the lowest income (the bottom

quintile). It is a measure of the inequality of income distribution.

Income deciles Income deciles groups are computed on the basis of the total

equivalised disposable income attributed to each member of the

household. Nine cut-point values (the so-called deciles cut-off

points) of income are identified, dividing the survey population

into ten groups equally represented by 10 % of individuals each:

The data (of each person) are sorted according to the value of the

total equivalised disposable income and then divided into 10 equal

groups each containing 10 % of individuals. For example, the first

decile group represents the 10 % of the population with the lowest

income and decile 1 is the cut-off point for this group. Five

representative income deciles have been selected in our analysis to

show the evolution of the different parts of the national income

distribution.

For more details on the calculation of the indicators please see

EU-SILC notes on the calculation of indicators.

Flash estimates should estimate to the extent possible the values captured in the EU-SILC6 survey.

The main target indicators (AROP and QSR) are based on an entire distribution that evolve relatively

slowly, except in times of crisis. Survey based yearly changes can be rather small and/or not

statistically significant. It is therefore relevant

to assess yearly changes together with the trends during a certain period across several years,

to read the whole set of indicators that provide a coherent picture about the evolution of the

underlying income7 distribution in each country. Deciles offer a complementary reading tool

in order to link the decrease of poverty or inequality with the relative movement at different

points of the distribution. Deciles can help in answering better policy questions like: is a

4 The equivalised takes into account the structure of the household. The income is calculated by dividing the total household

income by its size determined after applying the following weights: 1.0 to the first adult, 0.5 to each other household

members aged 14 or over and 0.3 to each household member aged less than 14 years old. 5 See for instance the Joint Report by the Commission and the Council on social inclusion as adopted by the Council

(EPSCO) on 4 March 2004, http://ec.europa.eu/employment_social/soc-prot/soc-

incl/final_joint_inclusion_report_2003_en.pdf 6 http://ec.europa.eu/eurostat/web/income-and-living-conditions/overview 7 http://ec.europa.eu/eurostat/statistics-explained/index.php/Glossary:Equivalised_disposable_income

http://ec.europa.eu/eurostat/statistics-explained/index.php/EU_statistics_on_income_and_living_conditions_(EU-

SILC)_methodology_%E2%80%93_concepts_and_contents

4

possible decrease of poverty related to a higher increase of the income for poorer people (left

tail of the distribution) or is a possible decrease linked to a decline of the middle class? More

generally, the examination of deciles at different points of the distribution helps to answer the

questions on who is benefiting from the growth and who is affected by the recession.

3. How are the flash estimates on income distribution produced?

The Flash Estimates should anticipate the changes (that will appear later in EU-SILC) based on

auxiliary information already available for the target year. Yearly changes are estimated as described

below and combined with the EU-SILC value for the preceding year, which constitutes the baseline

for the analysis.

A variety of approaches were tested, tailored to each country situation and we have selected the most

robust methodology for a given country. The publication as experimental statistics puts the basis for

receiving feedback from users and the research community and further improving the flash estimates.

The main methodology used for most countries is Microsimulation. It relies on EUROMOD, the

European Union tax-benefit microsimulation model, managed, maintained and developed by the

Institute for Social and Economic Research (ISER) at the University of Essex. For the purposes of the

flash estimates exercise standard EUROMOD policy simulation routines are enhanced with additional

adjustments to the input data to take into account changes in the population structure, the evolution of

employment and main indexation factors. The microsimulation approach in the frame of the flash

estimates exercise is based on previous work done by ISER, University of Essex (Rastrigina, O.,

Leventi, C., Vujackov S. and Sutherland, H. (2016)) and is being further developed by Eurostat in

collaboration with the dedicated Task Force on “Flash estimates on income distribution”. In general,

microsimulation is the preferred approach for both main users and the National Statistical Institutes

(NSIs) given the possibilities for further detailed analyses and the link with policy changes.

For few countries for which microsimulation results were not consistent enough with EU-SILC, a

second methodology is used based on macro-economic time series modelling (METS) with which the

model attempts to nowcast the indicators directly, based on linear regression, with extensions taking

into account the time series nature of the data.

Table 2 summarises the methodological approach chosen by country. Eurostat has produced flash

estimates for 25 countries. For Denmark8 and the Netherlands

9, provisional national register data were

used. Flash estimates for the United Kingdom are not produced as data for income 2016 is already

published10

in EU-SILC.

8 For Denmark, the definitions of the income match the EU- SILC, but there may be small differences in the household

definitions. There is no guarantee for a perfect match with the final SILC caused by statistical uncertainty due to sampling

in SILC compared to the full population in the register. 9 For the Netherlands, the definition of equivalised income is almost equal to the EU-SILC definition except for the inter-

household transfers which are not included. The inter-household transfers form only a small part of the total income, so the

deciles in both statistics are quite comparable. In general, inter-household transfers are paid by the higher income groups,

so the upper deciles may be somewhat actually lower in EU-SILC compared to the national income statistics. 10 SILC 2016 for UK covers current income, therefore, the income reference period is also 2016

5

Table 2. Methodological approach by country

Methodological approach Countries

Microsimulation BG, DE, EE, EL, ES, FR, HR, IT, CY, LV, LT, LU, HU, MT, AT,

PL, PT, RO, SI, SK, FI, SE

METS BE, CZ, IE

National provisional data DK, NL

An essential point in this exercise was the active participation of the Member States at different levels

and the support from the academic community, in particular the University of Essex, in the validation

and improvement of the FE methodology and of the flash estimates 2016. For more details please

consult also the Methodological Note including the description of both microsimulation and METS.

4. How where the flash estimates assessed?

Flash estimates income 2016 are produced by Eurostat (unless specified differently) and published as

experimental statistics.

The publication as experimental statistics puts the basis for receiving feedback from users and the

research community and further improving the flash estimates. However, the accuracy of the

indicators depends on the model assumptions and on several factors explained throughout the quality

assessment. As with any other flash estimate, we cannot expect to capture perfectly changes in the

SILC estimates. Differences can emerge, due to inconsistencies in the input datasets, model errors or

theoretical assumptions underlying the microsimulation techniques.

Developing flash estimates on poverty and income inequalities in the ESS involves that their methods,

sources and output adhere to a common quality framework. This was developed together with the

Member States and validated with the National Statistical Institutes and the academic community.

The quality framework contains two main parts:

1. Quality as an integrated process in the production: Ensure that quality is considered in the

inputs and methods used in all the steps of the production, by analysing inconsistencies in the

input data and performing several intermediate quality checks along the process. It is useful

for identifying possible sources of error and ways of for fixing them.

2. Quality assessment put in place in order to ensure a comparable way to assess results

stemming from different methods and national estimates within this ESS exercise:

2.1 the historical performance of the model is defined as the ability to predict accurately the

past changes in the main target indicators as captured by EU-SILC. Flash estimates were

simulated from 2012 to 2015 and compared with EU-SILC indicators.

2.2 the plausibility of the estimated change is assessed based on the available information

for the target year. Connecting the estimated changes in the income distribution with

observed evolutions in related indicators (e.g. employment trends, total household

6

income in national accounts, national data) is a key step in building trust in the

estimates. A trusted estimate is a reasonably good stand-in, to be used for drawing

preliminary conclusions until actual data becomes available. Unlike forecasting, for

flash estimates we have several auxiliary sources in the target year which are used either

in the estimation process or for validation checks (for plausibility assessment).

Furthermore, in the case of microsimulation we can disentangle the impact of simulated

policies via EUROMOD and supported with ISER, University of Essex analysis11

.

11 EUROMOD (2017) "Effects of tax-benefit policy changes across the income distributions of the EU-28 countries: 2015-

2016", EUROMOD Working Paper 10/17, Institute for Social and Economic Research, University of Essex

7

5. Flash estimates: communication on magnitude and direction of change

This report presents the figures for the flash estimates relating to the income year 2016 (FE 2016, i.e.

SILC 2017 whose results are expected in summer 2018 for most countries).

Flash estimates refer to the year-on-year (YoY) change rather than the actual levels and they are

calculated as the difference between the model estimates for 2016 and 2015 (or modelled directly in

the case of METS). The point estimate for the flash is calculated by adding the estimated YoY change

to the last observed value in EU-SILC.

Point estimates are subject to several sources of uncertainty: e.g. model bias and variance, the

sampling error in EU-SILC, inconsistencies between the different data sources entering the

estimation. This raises not only a question of quality, but also of communication of the results.

Following in-depth discussions with both users and producers, it was decided that results for this

release of the flash estimates are disseminated according to a magnitude direction scale (MDS) for

the expected change (major/moderate/minor). This dissemination format takes into account that

estimated changes cover a possible range of values, associated with uncertainty12

. Therefore, the FE

will give an indication on the type of change expected in terms of intervals, but not the point

estimates. The MDS is based on absolute thresholds, common across countries and indicator specific,

which makes it straightforward to understand. These were selected based on average values for

changes observed in EU-SILC in the past and in agreement with the Member States. They take also

into account similar exercises13

by policy makers which use thresholds to identify "substantive"

changes. The table below shows the MDS classes chosen by indicator and common across countries.

Table 3. Bounds MDS by indicators (different colour coding by group of indicators for

readability)

MDC AROP(pp) QSR Deciles (%)

Decrease in inequality

indicators=green14

Increase in positional

indicators = green

[---] <-2 <-0.6 <-5

[--] [-2,-1[ [-0.6,-0.3[ [-5,-2[

[-] [-1,0] [-0.3,0] [-2,0]

[+] ]0, 1] ]0, 0.3] ]0, 2]

[++] ]1,2] ]0.3,0.6] ]2,5]

[+++] >2 >0.6 >5

In addition, the magnitude direction class includes also a condition for statistical significance: only

the statistically significant changes are published. At this stage, only the sampling error is considered

for the significance of the change. This translates into a country specific additional "not significant

change" class centered around zero. This implies also that many of the changes mainly in the "minor"

change classes above ([-],[+]) will be not significant. More details are given in Annex 1.

12 UNSD - Handbook on Rapid Estimates (Draft for Global Consultation, 28 October 2016) 13 SPC publications 14 The colour coding used for the QSR is there for readability purpose and does not imply judgements about the desirable

level of inequality over the whole distribution.

MDC (change class)

communicated only if

the estimated change

is significant

8

The main advantages of the approach chosen are to guide the reader, in terms of statistical

significance (to avoid over-interpretation of non-significant changes) and to provide useful

information for users and policy makers concerning the expected changes and trends for income

indicators. There are however still limitations in the current format of the MDS. In particular,

imposing predetermined absolute thresholds for delimitating the type of change. At this stage this

doesn't allow to take into account the probability to be in a specific change class. It is important to

further discuss with both producers and users about the trade-offs of different communication options.

6. Income evolution in 2016: flash estimates

This section presents the figures for flash estimates 2016 in terms of absolute change for AROP and

QSR and change in percent for the deciles. Table 4 below shows the FE 2016 (and the FE 2015 when

EU-SILC 2016 (2015 income) data not available yet15

namely CY, DE, FR, MT, IT) translated into

the magnitude direction scale (MDS). Please note that only those estimates indicated as fit-for-

purpose, meaning passing the quality assessment framework, are disseminated. The UK data provided

is based on EU-SILC 2016 as UK collects current income so the data collection and the income

reference period are the same.

In table 4, the first two columns show the change in income inequalities, measured by the change in

percentage points (pp) in AROP and QSR, between 2015 and 2016 for all countries and also between

2014 and 2015 for the above mentioned countries. The third to seventh column show the percentage

change (%) in the income deciles 1, 3, 5 (or MEDIAN), 7 and 9. Taking first the example of Spain,

we observe a moderate increase (between 2% and 5%) of the decile 3, the median and the deciles 7

and 9. The evolution of the decile 1 is unreliable (does not pass the quality assessment). No significant

changes are predicted for the AROP and the QSR. Taking the example of Portugal, decile 3 to 9 are

likely to have an increase between 2% and 5%, but the increase of D1 is higher than 5%. This

translates into the estimated decrease of AROP and a non- significant change in QSR.

The change in income inequalities between 2014 and 2015 is also available for the countries for

which EU-SILC 2016 was not available at the time of this publication. For instance, for Germany,

while the AROP is estimated to decrease up to 1pp between 2015 and 2016, the estimated change in

AROP between 2014 and 2015 is not significant.

It is good to note that an estimated non-significant change can be a strong message. In the case of

Cyprus, for instance, these non-significant changes over two years could indicate that the declining

trend of the income at all levels of the distribution observed in the income years 2012-2014 has

stopped.

Nominal values are used in this table. Deflated data are available in Annex 2.

15LU FE income 2015 are not included even if EU SILC 2016 (income 2015) is not available due to a break in series in LFS.

Therefore it was not possible to accurately estimate changes in employment.

IE FE 2016 only because it collects the income reference period is the last 12 month from the interview.

9

Table 4. Flash estimates of the nominal change 2015-1616

- published as experimental data

under the responsibility of Eurostat, unless otherwise specified

Country Income Year AROP QSR D1 D3 MEDIAN D7 D9

BE 2016 u ]0,0.3] ns u ]2,5] u ]2,5]

BG 2016 ns ns >5 >5 >5 >5 >5

CZ 2016 ns ns >5 >5 >5 >5 u

DK* 2016 ns ns ns ns ns ns ns

EE 2016 ns ns >5 >5 >5 >5 >5

IE 2016 ns [-0,6,-0,3[ >5 ]2,5] u >5 ]2,5]

EL 2016 ns ns ]2,5] u ns u u

ES 2016 ns ns u ]2,5] ]2,5] ]2,5] ]2,5]

HR 2016 ns ns ns ]2,5] ]2,5] ]2,5] ]0,2]

LV 2016 ns ns ]2,5] ]2,5] ]2,5] ]2,5] ]2,5]

LT 2016 u ns >5 >5 >5 >5 >5

LU 2016 ns ns ns ns ns ns ns

HU 2016 ns ns >5 >5 >5 >5 >5

NL* 2016 ns ns ]0,2] ]2,5] ]2,5] ]2,5] ]2,5]

AT 2016 ns ns ns ]2,5] ]2,5] ]2,5] u

PL 2016 <-2 <-0.6 >5 >5 >5 >5 >5

PT 2016 [-1,0] ns >5 ]2,5] ]2,5] ]2,5] ]2,5]

RO 2016 ]0,1] ns ns >5 >5 >5 >5

SI 2016 [-1,0] ns ]2,5] u ]0,2] u u

SK 2016 ns ns u u ]2,5] u u

FI 2016 ns ns ns ]0,2] ]0,2] ]0,2] ns

SE 2016 u u u u ]2,5] ]2,5] ]2,5]

UK 2016 [-1,0] ns ns ]2,5] ]2,5] ]0,2] ]0,2]

DE 2015 ns u u ]0,2] ]0,2] ]0,2] ns

DE 2016 [-1,0] u u ]0,2] ns ns ns

FR 2015 ns ns ns ]0,2] ]0,2] ns ns

FR 2016 ns ns ns ns ]0,2] ]0,2] ns

IT 2015 ns ns ]2,5] ]0,2] ]0,2] ns ns

IT 2016 ns ns ]2,5] ]0,2] ]0,2] ]0,2] ns

CY 2015 u ns ns ns ns ns ns

CY 2016 u ns ns ns ns ns ns

MT 2015 ns ns ns ]2,5] ]2,5] ]2,5] ]2,5]

MT 2016 ns ns ]2,5] ns ns ns ns

*National preliminary results are used;

u: unreliable, ns: not significant

Source: Eurostat calculations based on EUROMOD H0.25+ and Eurostat data sources (EU-SILC, LFS, Sector Accounts)

16 Please see also Table 5 in Annex 2

10

The results obtained are also presented by country and by indicator over time, in an additional

document, so as to help the reader in assessing trends in the data.

7. Some main messages for the FE 2016

In terms of main messages that we can draw based on the flash estimates we can notice that:

In general, the FE 2016 show an overall increase of the equivalised disposable income across

the distribution for almost all countries. These estimated changes are supported by main

trends in employment situation including average increases in wages, as well as the evolution

of the gross disposable income in Sectoral Accounts.

We can also observe that AROP and QSR for the majority of countries have not significantly

changed. This is in line with previous developments of these indicators in EU-SILC when

most of the yearly changes are not significant. In general, it can be interpreted as a status quo.

In some cases, it has to be interpreted with caution because it can be rather the result of a

large standard deviation for the specific country.

The joint analysis of deciles yearly changes provides also more information on the evolution

at different points of the distribution. The main differences come from the relative movement

of the left part in relation to the middle/upper part of the distribution.

It is also very important to look at the flash estimates together with the time series in EU-

SILC across several years for all indicators in order to assess the yearly changes.

These messages can be further detailed for individual countries.

The estimated change in the AROP between 2015 and 2016 is statistically significant for 5 countries:

Romania, Poland, Portugal, Slovenia and Germany. The only country for which the AROP is

estimated to increase is Romania. This is related to the increase larger than 5% which is estimated for

all deciles except for D1.

The largest decrease in AROP is estimated for Poland. An increase larger than 5% is estimated for all

deciles which translated into a significant decrease in the estimated AROP and QSR. The magnitude

of this decrease should be read with caution. This change is predicted by the model in relation to the

introduction of means-tested benefits and relies on the assumption that there was full benefit take up.

There are also significant decreases estimated in the AROP for Portugal, Slovenia and Germany.

Overall, they were caused by a larger increase estimated in the left part of the income distribution than

in the rest of the distribution. For Belgium, a significant increase and for Ireland a significant

decrease is to be noticed for QSR. For Germany, there is a significant decrease estimated for AROP

between 2015 and 2016 while between 2014 and 2015, a more similar change across deciles is

estimated which didn't impact the AROP.

For the rest of the countries, while the estimated change in the AROP is not significant, there were

statistically significant changes in the estimated income deciles. A similar increase across most

deciles is estimated for 7 countries, with an estimated increase larger than 5% for Bulgaria, the

Czech Republic, Estonia, Hungary and Lithuania and an estimated increase between 2% and 5%

for Spain and Latvia.

11

There are 7 countries with different estimated increases across the distribution that didn't impact the

inequality indicators. For Croatia, Finland, France, Netherlands and Austria, there were mostly

similar increases estimated except in the tails (which can be due either to lower estimated increase in

the tail or higher variance for the extreme deciles). For Croatia, the middle deciles are estimated to

increase between 2% and 5% between 2015 and 2016, which is more than the tails (D1 and D9).

There were increases of up to 2% estimated for Finland and France. For Austria, the middle deciles

(D3, MEDIAN and D7) are estimated to increase significantly but not D1 (D9 is not reliable).

A larger change in D1 than in the other part of the distribution is estimated for Malta, Greece and

Italy. For Malta, in 2016, the estimated change in D1 is significant but not the other deciles. In 2015,

all the deciles except D1 are estimated to increase significantly. For Greece, there is a larger increase

estimated in D1 than in the MEDIAN.

For Cyprus, Denmark and Luxembourg there are no significant changes estimated in any of the

indicators. The non-significance can be interpreted in general as a status quo17

. This is also the case

for Cyprus in 2015. For Luxembourg, FE 2015 were not estimated due to a break in series in LFS

2015.

Finally, for Belgium, Sweden and Slovakia, the estimated changes in the income deciles are reliable

only for some indicators. For Sweden, a similar increase in the mid to right part of the distribution is

estimated while for Slovakia, there is an increase between 2% and 5% estimated in the MEDIAN.

From a policy point of view and for microsimulation countries, estimated changes are driven by

several elements that enter the nowcasting model: changes in employment situation and the socio-

demographic structure, change in the evolution of different income components and the impact of

simulated policies. The impact of policies is calculated on the base of EUROMOD policy tool and is

supported with the information collected through the EUROMOD network.18

The models assume

most of the time that these policies are actually implemented.19

Further analysis is therefore possible.

17 To note that in some cases the non-significance can also result from large standard deviations for the specific country (LU) 18 EUROMOD (2017) "Effects of tax-benefit policy changes across the income distributions of the EU-28 countries: 2015-

2016", EUROMOD Working Paper 10/17, Institute for Social and Economic Research, University of Essex 19 Tax evasion (e.g. in Bulgaria, Greece, Italy and Romania) and benefit non-take-up (e.g. in Estonia, France, Greece, Latvia,

Romania, and Finland)

12

8. How to improve the flash estimates?

This is the first publication of flash estimates on income distribution as experimental statistics. The

report contains not only the estimated changes for the target year but also a few elements on the

estimation process, auxiliary sources used to support the analysis of the figures and their reliability. It

is meant to put the basis for a constructive dialogue for further improving the methodology and the

dissemination of these indicators.

To help Eurostat improve these experimental statistics, users and researchers are kindly invited to give

us their feedback:

Would you have comments or suggestions for improvements of the methods applied for this

flash estimate exercise, i.e. based on either microsimulation or METS?

Are there any other factors we should consider?

What other indicators or breakdowns could be useful as early warnings on trends in income

distribution and poverty?

Are there other indicators we should analyse for policy purposes?

Is the magnitude direction scale clear and easy to understand? How to improve it? Would

point estimates be desirable in the future?

Further developments could be envisaged, following also the feedback from users and stakeholders:

improve the dissemination format including the significance and the magnitude-direction

scale;

use of more recent EU-SILC files for microsimulation so that to minimize the impact of

revisions and breaks in series but as well to improve the model;

further develop the plausibility assessment on the line of disentangling the impact of

employment, uprating factors and policies on each indicator;

better take into account the coherence of the estimated changes across the distribution

(especially for METS).

13

Annex 1: Magnitude direction scale and significance

As mentioned the MDS is based on absolute thresholds, common across countries and indicator

specific. It is important to note that the actual MD class will be communicated only if the change is

statistically significant. At this stage, the sampling error is considered for the significance of the

change and is country specific. In specific countries, with large standard deviations, higher values of

yearly changes will be considered not statistically different from zero.

For the main inequality indicators we have used the usual calculation of Eurostat for the standard

deviation of the net change20

. It calculates the variance of the net change based on multivariate linear

regression technique (Berger and Priam , 2016) that reduces non-linear statistics to a linear form and

takes into account the overlap of samples between years. For deciles we have developed a

bootstrapping procedure for computing the variance of the estimates. We have used 1000 subsamples

of the SILC dataset at the target year with each individual having a probability of wj

∑ wjpj=1

to be drawn

where wj denotes the sample weight of the jth individual and the size of the subsamples being equal to

the number of individuals in the SILC dataset . Then we compute all indicators of interest for each

one of these replicated data sets. The collection of computed indicators can then be used to obtain an

estimate of the sampling distribution of the SILC indicators (unweighted). The standard deviation of

the change for deciles is likely to be overestimated as it doesn't consider the overlap of samples

between two consecutive years in EU-SILC. In the future, it is foreseen to apply the same estimation

procedure as for AROP and QSR.

Figure 1 below shows the thresholds for the MD classes, including the significance bounds for AROP

and Median for all countries. It shows also that when we overlap the significance bounds (ns series),

these vary considerably across countries. Therefore, for example a change of 1 percentage point for

AROP will be communicated for certain countries at the right as not significant (ns), while for other

countries with lower SD (at the left of the graph), it would be a change in the interval ]0, 1] pp. It is

very important to note that most of the figures in the minor change classes will be not significant so

no MD class will be communicated.

20 http://ec.europa.eu/eurostat/documents/3888793/5855973/KS-RA-13-024-EN.PDF/cfef2973-4675-4df4-bf6d-

e15ef1d3c060

14

Figure 1: Thresholds for the magnitude direction scale, including the significance bounds

In conclusion, the MDS has advantages related to the communication with users: 1) it takes into

account the uncertainty inherent in our estimates and the input data, avoiding interpretation of changes

not statistically significant and 2) provides a straightforward way of communicating the direction and

magnitude of the change to the users. However, it is important to note two important drawbacks of the

MDS:

-3.00

-2.50

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

BE

UK

DE

FR PT

RO SI EL ES CZ

EE FI SE BG

HR IT LV PL

AT

CY

DK

MT

NL

SK IE

HU LT LU

AROP: MD classes

ns [-] [+] [--] [++] [---] [+++]

-8%

-6%

-4%

-2%

0%

2%

4%

6%

8%

LT SI IT FR NL FI DE

UK PT EL PL

HU SK CZ

BE ES DK SE BG AT

RO

HR

LU CY LV MT IE EE

MEDIAN: MD classes

ns [-] [--] [+] [++] [+++] [---]

15

The MDS, unlike a confidence interval it is not centred on the point estimates and therefore in

specific cases the estimates can be very close to the thresholds between MD classes therefore

increasing the likelihood that the final estimate can be in the adjacent category. No probability

of belonging to the MD class is associated for the moment.

For the computation of the significance further sources of uncertainty should be considered;

i.e. model bias and model variance. The bootstrapping procedure could incorporate different

sources of uncertainty related to the model through replication. However this has a high

computational intensity and doesn’t take into account the sample design and the overlap

samples between two cross sectional waves. Estimating the model bias is particularly difficult

due to the short time series. Further work will focus on improving the estimation of the

significance and the way is taken into account in the MDS.

16

Annex 2 Flash estimates of the real change 2015-16(deflated with HICP21)

Table 5: Flash estimates of the real change 2015-16 (deflated with HICP)

Country Income Year D1 D3 MEDIAN D7 D9

BE 2016 ns u ]0,2] u ]0,2]

BG 2016 >5 >5 >5 >5 >5

CZ 2016 >5 ]2,5] ]2,5] >5 u

DK* 2016 ns ns ns ns ns

EE 2016 >5 >5 >5 >5 >5

IE 2016 >5 >5 u >5 ]2,5]

EL 2016 ]2,5] u ns u u

ES 2016 u ]2,5] ]2,5] ]2,5] ]2,5]

HR 2016 ns ]2,5] ]2,5] ]2,5] ]2,5]

LV 2016 ]2,5] ]2,5] ]2,5] ]2,5] ]2,5]

LT 2016 >5 >5 >5 >5 >5

LU 2016 ns ns ns ns ns

HU 2016 >5 >5 >5 >5 >5

NL* 2016 ]0,2] ]2,5] ]2,5] ]2,5] ]2,5]

AT 2016 ns ]2,5] ]2,5] ]2,5] u

PL 2016 >5 >5 >5 >5 >5

PT 2016 >5 ]2,5] ]2,5] ]2,5] ]2,5]

RO 2016 ns >5 >5 >5 >5

SI 2016 ]2,5] u ]0,2] u u

SK 2016 u u ]2,5] u u

FI 2016 ns ]0,2] ]0,2] ]0,2] ns

SE 2016 u u ]2,5] ]2,5] ]0,2]

UK 2016 ns ]2,5] ]0,2] ]0,2] ]0,2]

DE 2015 u ]0,2] ]0,2] ]0,2] ns

DE 2016 u ]0,2] ns ns ns

FR 2015 ns ]0,2] ]0,2] ns ns

FR 2016 ns ns ]0,2] ]0,2] ns

IT 2015 ]2,5] ]0,2] ]0,2] ns ns

IT 2016 ]2,5] ]0,2] ]0,2] ]0,2] ns

CY 2015 ns ns ns ns ns

CY 2016 ns ns ns ns ns

MT 2015 ns ]0,2] ]0,2] ]0,2] ]2,5]

MT 2016 ]0,2] ns ns ns ns

*National preliminary data is used;

u: unreliable, ns: not significant

Source: Eurostat calculations based on EUROMOD H0.25+ and Eurostat data sources (EU-SILC, LFS, Sector Accounts)

21 http://ec.europa.eu/eurostat/web/hicp

17

Annex 3. Quality Assessment Framework

Flash estimates are assessed on a specific quality framework developed together with the Member

States and validated via a dedicated Task Force with the National Statistical Institutes and the

academic community. This QAF aims to provide a common platform to assess Eurostat and

national estimates.

The quality framework is composed of two parts: (1) the quality assurance, which focuses on

analysing inconsistencies in the input data and includes several intermediate quality checks along the

process; (2) the quality assessment, which focuses on the historical performance of different methods.

Quality Assurance

The quality framework is an essential tool for designing the production process of flash estimates.

Therefore, the quality framework doesn't focus only on the final results but includes the inputs and

methods used in all the steps of the production, by analysing inconsistencies in the input data and

performing several intermediate quality checks along the process. It is useful to identify possible

sources of error and ways of fixing them. It is also an essential tool for designing the production

process of flash estimates. For example, employment trends as measured by LFS or simulated benefits

via EUROMOD are compared to EU-SILC statistics for the past. The lack of such consistency could

have an important impact on the historical performance of the model.

Quality Assessment

There are two main dimensions that were used for the decision to publish the FE 2016: 1) the

historical performance of the model defined as the ability to retropredict accurately changes in all

main indicators as captured by EU- SILC. Flash estimates were simulated from 2012 to 2015 and

compared with EU-SILC indicators , and 2) the plausibility of the estimated change assessed via

several elements: the evolution of related indicators used in the estimation (e.g. employment, social

benefits and taxes simulated via microsimulation); consistency with similar income statistics at

aggregated level in sectoral accounts; and subjective indicators in EU_SILC.

1) Historical performance

The historical performance is mainly assessed based on mean absolute error (MAE- for the percentual

change)22

& (MAE- for the absolute change)23

. This is supported by a much more detailed analysis of

income components and labour variables. The analysis of historical performance is based on

simulations of the flash estimates from 2012-2015. The estimated year-on-year change (YOY_EST)

for the years 2012 to 2015 is compared with the year-on-year change (YOY_REF) for SILC 2013 to

22 𝑀𝐴𝐸 = 𝑚𝑒𝑎𝑛(|𝑒𝑦|) 𝑤ℎ𝑒𝑟𝑒 𝑓𝑜𝑟 𝑑𝑒𝑐𝑖𝑙𝑒𝑠, 𝑒𝑦 = 𝑌𝑜𝑌. 𝑅𝐸𝐹𝑦 − 𝑌𝑜𝑌. 𝐸𝑆𝑇𝑦(𝑜𝑟 𝑌𝑜𝑌) =

𝑅𝐸𝐹𝑦

𝑅𝐸𝐹𝑦−1−

𝐸𝑆𝑇𝑦

𝐸𝑆𝑇𝑦−1

23 𝑀𝐴𝐸 = 𝑚𝑒𝑎𝑛(|𝑒𝑦|) 𝑤ℎ𝑒𝑟𝑒 𝑓𝑜𝑟 𝐴𝑅𝑂𝑃 𝑎𝑛𝑑 𝑄𝑆𝑅, 𝑒𝑦 = (𝑅𝐸𝐹𝑦 − 𝑅𝐸𝐹𝑦−1) − (𝐸𝑆𝑇𝑦 − 𝐸𝑆𝑇𝑦−1)

18

SILC 2016. In this assessment we also take into account the standard deviation of the target

indicators: the lower the variance of EU-SILC indicators, the more stringent we are with the

thresholds for MAE as we consider the points estimates should be close. When the confidence interval

for the target indicator for a specific country is larger, the quality requirements are more lenient: the

FE is still considered fit for purpose even if the points estimates are not very close but still in the

confidence interval.

In general, results for the microsimulation when simulating back based on older files (2012 SILC) can

be affected by breaks in SILC data series and revisions. Results improve for the last years, as more

recent files are used for producing the flash estimates and with ongoing efforts to introduce

disaggregated benefits in EU-SILC and to improve the precision of simulations in EUROMOD. Table

6 summarises the main conclusions that we can draw concerning the historical performance across

indicators with three main categories of countries.

Table 6. Assessment of the reliability of the estimates based on past performance for the period

2012-2015*

Countries Assessment

BG, FR, LU, HU, MT,

PL, FI

All indicators have a relatively low average past error, or with few

exceptions in the right part of the distribution (PL, MT)

CZ, DE, EE, ES, HR,

IT, CY, LV, LT, AT,

PT, RO

All indicators have a relatively low average past error, but in general

there are some problems in capturing the extreme deciles. In some cases

(CY, DE, LT) this impacts on the performance of the model for the

inequality indicators.

BE, IE, EL, SI, SK, SE The model has a relatively low error for the inequality indicators (AROP,

QSR) but not for the deciles.

*for CY, IT, LU, DE, FR, MT the period is 2012-2014. The assessment of the error is function also of

the standard deviation of the indicator by country.

2) Plausibility

There are four main parts in the plausibility analysis:

1) An analysis of the plausibility of the FE given the general evolution for related indicators

(employment, wages, social benefits) including the impact of simulated policy changes

calculated using the SILC 2015 file and the EURMOD model, supported with the analysis 24

of ISER, University of Essex (microsimulation countries only);

2) A comparison with the National Accounts data for gross disposable income and main income

components at aggregated level (microsimulation countries only);

3) A comparison with main SILC indicators which capture more distributional aspects

4) Additional national information provided by Member States (where available).

24 EUROMOD (2017) "Effects of tax-benefit policy changes across the income distributions of the EU-28 countries: 2015-

2016", EUROMOD Working Paper 10/17, Institute for Social and Economic Research, University of Essex

19

1) Table 7 illustrates the main elements from the target year used for 'updating' which is combined

with the latest EU-SILC information and enter the microsimulation model. Therefore, the third

column contains the bar charts for the evolution of the deciles (in terms of MD class). The following

columns should be read as a map for assessing the plausibility of estimated changes for the deciles at

different points of the distribution. It is important to make the difference between two main types of

changes:

a) observed changes observed in auxiliary sources (EU-LFS and national sources) and which are used

for "updating" the labour and employment income. At this stage, these are not yet translated in

specific contributions in the estimated changes;

b) the last three columns refer directly to the impact of simulated social benefits and taxes via

EUROMOD on the average disposable income (keeping the socio demographic structure and market

incomes constant). This impact is also further detailed in terms of: main benefits that have an impact

for each country and charts showing the simulated impact differentiated across the distribution. The

latter is based on the calculated impact of simulated policies on the average income across decile

groups in the distribution.

The changes are expressed in terms of intervals and it shows just the direction and magnitude of

changes for each factor. This is a first step towards providing to users information on the main factors

that enter the model and lead to the estimated changes. For example, there are several countries where

the uprating factors applied to update the employment income are rather large and seem to have a

positive effect on the estimated changes for deciles (e.g. RO, BG, EE, LT). In terms of relative

movement of the deciles, there are usually the simulated policies via Euromod that seem to have a

differential impact across the estimated distribution25

(e.g. DE, PL, PT, RO, SI). It is also important to

note that the microsimulation of social benefits and taxes relies on some theoretical assumptions

concerning the implementation of policies. For example, in PL the means-tested benefits could be

oversimulated which could lead to overestimation of the increase in D1 and the major decrease of

inequality indicators.

Further work should focus on disentangling the contribution of each factor in the estimated changes

for our target indicators, including the contribution from employment updates, other income

components or simulated policies.

25 Eurostat calculations based on EUROMOD

20

Table 7. Evolution of labour market, employment income and impact of policy changes262728

*MDS (%change)

26 Policies: "-" (<0.5%); " =" between (-0.5,0.5);"+" (>0.5%); "++"( >2%); "+++"( >5%); 27 Employment: "-" (<1%); " =" between (-1,1);"+" (>0.5%); "++"( >3%); "+++"( >5%); 28 Uprating factors: "-" (<0.5%); " =" between (-0.5,0.5);"+" (>0.5%); "++"( >2%); "+++"( >5%);

% change

Employment 35

% change

Uprating factor

income from

employment)36

Simualted impact

on the % change of

average disposable

income

Simulated

impact across

distribution

BG 2016 = +++ +

DE 2016 + +/++ +

EE 2016 = +++ ++

EL 2016 + +--

ES 2016 + - =

FR 2016 = + =

HR 2016 = + =

IT 2016 + = =

CY 2016 + - =

LV 2016 = ++ =

LT 2016 + +++ +

LU 2016 + = =

HU 2016 ++ ++ +

MT 2016 ++ = =

AT 2016 + + ++

PL 2016 = ++ +++

PT 2016 + = / + +

RO 2016 = +++ ++

SI 2016 = + =

SK 2016 + ++ =

FI 2016 = + =

SE 2016 + + =

CountryIncome

Year

FE DECILES

(MDS*)

D1 D3 D5 D7 D9

Observed changesImpact of simulated social benefits

and taxes34

<-5 [-5,-2[ [-2,0[ ns/= ]0, 2] ]2,5] >5

21

2) Table 8 provides a comparative change of the MD class for the yearly change of the total

disposable income between the FE and Quarterly Sector Accounts29

. We include only countries for

which quarterly data is available for the sector household; non-profit institutions serving households

(S14_S15). This should be read taking into account the underlying comparability of income (trends)

from EU-SILC and National Accounts. For more details on the latter please see also Gregorini et al,

(2016)30

. If we apply the MDS for changes in total disposable income, we notice that the trends and

magnitude are similar in the target year.

Table 8. Comparison with National accounts: evolution total disposable income

COUNTRY MD class total income FE/NA31

PL, RO +++/+++

DE, ES, SE, SI, PT ++/++

AT +++/++

FI, FR, IT, EL =/=

3) The additional SILC variables selected for assessing plausibility are NOT used for producing the

estimates; they are:

Ability to make ends meet (MEM) = % HS120=4÷6 (fairly easily, easily, very easily)

Severe Material Deprivation (SMD) = % Severe Material Deprivation=1

Capacity to face unexpected financial expenses (UXX) = % HS060=1 (yes)

We have used these additional variables to calculate the so-called "external estimates" from 2012

onwards, using a rolling data set. Values of the external estimate are based on a univariate linear

regression, using in turn MEM, SMD, and UXX as predictors. We have also calculated the prediction

interval of the external model, which takes into account only the uncertainty generated by the

estimation model, but not that already present in the input data.

Potential further work might be necessary, insofar as an assessment based on the external estimates

does not allow us to draw strong conclusions regarding the plausibility of the flash estimates, because

the prediction intervals of the external estimates are usually very large, and cover a large part or even

the entire possible range of variation. We are aware of new avenues for assessing the plausibility of

the FE, especially some focused on the coherence of the estimated changes for indicators coming

from the same distribution.

4) In addition to the aforementioned plausibility analysis, all Member States were consulted

concerning the flash estimates and in some cases we have received additional information based on

national sources or models.

29 Source: Eurostat calculations- gross disposable income [nasq_10_nf_tr] 30 http://www.iariw.org/dresden/gregorini.pdf 31 FE/ NA growth rate disposable income ("=" means between -2% and 2%;"++" means between 2% and 5%; "+++" means

>5%)

22

Annex 4 - Data sources and availability

The data used in this report for the flash estimates is based on Eurostat estimations. The information

set that entered the estimation includes 1) for microsimulation the EUROMOD model combined with

the latest EU-SILC users' database (UDB) microdata file and/or national SILC microdata32

available

at the time of production33

2) for macro-economic time series modelling (METS) it includes the time

series for EU-SILC until the latest year available34

. This is enhanced with more timely auxiliary

information from the reference period (2016) such as Labour Force Survey (LFS), National Accounts,

etc.

The data used for the target indicators for the year (2012-2015) are primarily derived from data from

EU statistics on income and living conditions (EU-SILC). The reference population is all private

households and their current members residing in the territory of an EU Member State at the time of

data collection; persons living in collective households and in institutions are generally excluded from

the target population.

Main tables

Income and living conditions (t_ilc)

EU-SILC further information

Income, social inclusion and living conditions

EU statistics on income and living conditions (EU-SILC) methodology

32 UDB EU-SILC 2015-1: BG ES FI HR HU LV PT SI

UDB EU-SILC 2015-2: CY EE FR LT LU MT PL RO SE

UDB EU-SILC 2014-1: DE

In addition, for EE LT LU PL LV SI FI, additional national SILC variables were also used

National SILC 2015: EL IT SK AT (+ UDB EU-SILC 2015-1) 33EU-SILC 2015 UDB except DE (2014). In the meantime EU-SILC 2016 is available for most countries but not yet the

UDB and the EUROMOD input file 34 EU-SILC 2009-2016: BE and CZ, EU-SILC 2009-2015: IE

23

Annex 5 – References

Avram, S., Sutherland, H., Tasseva, I. and Tumino, A. (2011) Income protection and poverty

risk for the unemployed in Europe, Research Note 1/2011 of the European Observatory on

the Social Situation and Demography, European Commission.

Banbura, M., Giannone, D., Modugno, M. and Rechlin, L. (2013) Now-casting and the real-

time data flow, ECB Working Paper Series, no. 1564, pp. 1-53.

Berger, Y.G., and Priam, R. (2016) A simple variance estimator of change for rotating

repeated surveys: an application to the European Union Statistics on Income and Living

Conditions household surveys, Journal of the Royal Statistical Society, Series A Statistics

in Society, vol. 179, issue , January 2016, pp. 251–272

Bourguignon, F., Bussolo, M. and Da Silva, L.P. (2008) The impact of macro-economic

policies on poverty and income distribution Macro-Micro Evaluation Techniques and

Tools, The World Bank and Palgrave-Macmillan, New York.

Brandolini, A., D’Amuri, F. and Faiella, I. (2013) “Country case study – Italy”, Chapter 5 in

Jenkins et al, The Great Recession and the Distribution of Household Income, Oxford:

Oxford University Press.

Brewer, M., Browne, J., Hood, A., Joyce, R., Sibieta, L. (2013) The Short- and Medium-

Term Impacts of the Recession on the UK Income Distribution, Fiscal Studies, 34(2):

179–201.

Deville, J.-C. (1999). Estimation de variance pour des staistiques et des estimateurs

complexes : linéarisation et techniques des résidus. Techniques d’enquête, 25, 219-230.

Deville, J.-C. and Särndal, C.-E. (1992). Calibration estimators in survey samling. Journal of

the American Statistical Association, 87, 376-382.

Essama-Nssah, B. (2005) The Poverty and Distributional Impact of Macroeconomic Shocks

and Policies: A Review of Modeling Approaches, World Bank Policy Research Working

Paper 3682, Washington, DC.

EUROMOD (2017) “Effects of tax-benefit policy changes across the income distributions of

the EU-28 countries: 2015-2016”, EUROMOD Working Paper 10/17, Institute for Social

and Economic Research, University of Essex.

Fernandez Salgado M., Figari, F., Sutherland, H. and Tumino, A. (2014) Welfare

compensation for unemployment in the Great Recession, The Review of Income and

Wealth, 60(S1): 177–204.

Figari, F., Iacovou, M., Skew, A. and Sutherland, H. (2012) Approximations to the truth:

comparing survey and microsimulation approaches to measuring income for social

indicators, Social Indicators Research, 105(3): 387-407.

Figari, F., A. Paulus and H. Sutherland (2015) “Microsimulation and Policy Analysis” in

A.B. Atkinson and F. Bourguignon (Eds.) Handbook of Income Distribution, Vol 2B.

Amsterdam: Elsevier, pp. 2141-2221. ISBN 978-0-444-59430-3

Figari, F., Salvatori, A. and Sutherland, H. (2011) Economic downturn and stress testing

European welfare systems, Research in Labor Economics, 32: 257-286.

Hjort, N. L and Claeskens, G. (2003) Frequentist model average estimators, Journal of the

American Statistical Association, 98: 879-899.

24

Gasior, K. and Rastrigina, O. (2017) Nowcasting: timely indicators for monitoring risk of

poverty in 2014-2016, Research Note 1/2016, Social Situation Monitor, European

Commission.

Immervoll, H., Lindström, K., Mustonen, E., Riihelä, M., & Viitamäki, H. (2005c). Static

data “ageing” techniques. Accounting for population changes in tax-benefit

microsimulation. EUROMOD Working Paper, EM7/05.

Immervoll, H., Levy, H., Lietz, C., Mantovani, D. and Sutherland, H. (2006) The sensitivity

of poverty rates in the European Union to macro-level changes, Cambridge Journal of

Economics, 30: 181-199.

Jara, X.H. and Leventi, C. (2014) Baseline results from the EU27 EUROMOD (2009-2013),

EUROMOD Working Paper EM18/14, Colchester: ISER, University of Essex.

Keane C., Callan, T., Savage, M., Walsh, J.R. and Timoney, K. (2013) Identifying Policy

Impacts in the Crisis: Microsimulation Evidence on Tax and Welfare, Journal of the

Statistical and Social Inquiry Society of Ireland, XLII: 1-14.

MacDonald, J. B. and Xu, Y. J. (1995) A generalization of the beta distribution with

applications, Journal of Econometrics, vol. 66, no. 1-2, pp. 133-152.

Makovec, M. and Tammik, M. (2017) Baseline results from the EU28 EUROMOD (2011-

2016), EUROMOD Working Paper 6/17, Colchester: Institute for Social and Economic

Research, University of Essex.

Massey, J. (1951) The Kolmogorov-Smirnov Test for Goodness of Fit, Journal of the

American Statistical Association, vol. 46, no. 253, pp. 68-78.

Matsaganis, M. and Leventi C. (2014) Poverty and inequality during the Great Recession in

Greece, Political Studies Review, 12: 209 -223.

Narayan, A. and Sánchez-Páramo C. (2012) Knowing, when you don't know: using

microsimulation models to assess the poverty and distributional impacts of

macroeconomic shocks, The World Bank, Washington DC.

Navicke, J., Rastrigina, O. and Sutherland, H. (2013) Nowcasting Indicators of Poverty Risk

in the European Union: A Microsimulation Approach, Social Indicators Research, 119(1):

101-119.

O'Donoghue, C.,Loughrey, J. (2014) Nowcasting in Microsimulation Models: A

Methodological Survey, Journal of Artificial Societies and Social Simulation 17 (4) 12.

Peichl, A. (2008) The Benefits of Linking CGE and Microsimulation Models: Evidence from

a Flat Tax Analysis, IZA Discussion Paper No. 3715.

Rastrigina, O., Leventi, C., Vujackov S. and Sutherland, H. (2016) Nowcasting: estimating

developments in median household income and risk of poverty in 2014 and 2015,

Research Note 1/2015, Social Situation Monitor, European Commission.

Rastrigina, O., Leventi, C. and Sutherland, H. (2015) Nowcasting: estimating developments

in the risk of poverty and income distribution in 2013 and 2014, Research Note 1/2014,

Social Situation Monitor, European Commission.

Sutherland, H. and Figari, F. (2013) EUROMOD: the European Union tax-benefit

microsimulation model, International Journal of Microsimulation, 6(1): 4-26.