Part 2

The recommendation

on processing scanner

data

Berthold FeldmannEurostat Unit C4

Price Statistics; Purchasing Power Parities; Housing Statistics

Scanner Data Workshop – Rome October 20152

Structure of the talk

The challenges

The two alternative approaches

What should be part of the recommendation?

Timetable

2

Scanner Data Workshop – Rome October 2015

Need for a recommendation

By 2020 more than half of all NSIs may use scanner data for supermarkets

More and more tests are performed to expand the use of scanner data to clothing, electronic goods and other areas

Recommendations are needed to help NSIs use scanner data in an efficient way to assure the comparability of the results across

Europe

A meeting 8th & 9th Sep. among the NSIs using scanner data was the starting point for this recommendation

3

Scanner Data Workshop – Rome October 2015

Challenges in using scanner data

Treating big data sets

Identification of the articles

Linking products (GTIN or internal shop codes) to ECOICOP

Product replacement in such a large data set

The low life expectancy of GTINs

New GTINs may be re-launches

4

Scanner Data Workshop – Rome October 2015

The two approaches

Over the past 15 years two alternative approaches in using scanner data as a source of HICP emerged The fixed basket approach (FBA) and The dynamic sampling approach (DSA)

The names may be slightly misleading since

In the FBA the basket also changes because of product replacement

In the DSA the sample (basket) overlaps from month to month by more than 80%

5

Scanner Data Workshop – Rome October 2015

How to come to recommendations

First proposals will be based on currently implemented methods

It will contain 'recipes' that currently work It will also contain warnings, i.e. lessons learnt in

the NSIs (don't do…) All drafting in close cooperation with all stake

holders

6

Scanner Data Workshop – Rome October 2015

Possible issues for the recommendation?

Both alternative approaches?

Automation of the processes?

Use GTINs or shop internal codes?

Mapping GTINs to COICOP?

Cut-off levels for insignificant products – filters?

Imputation of missing prices?

Handling seasonal goods?

What other questions are missing?

7

Scanner Data Workshop – Rome October 2015

Tentative timetable

A first draft of the recommendation(for supermarkets) March 16

First discussion in the TFQI Spring 16

Discussion in the PSWG November 16

Extension to clothing, electronic goodsand others 2017

8

Scanner Data Workshop – Rome October 2015

Discussion in the same 6 groups

• What topics are missing?

• What topics can be dropped?

• What should be the main focus?

Both approaches?

Process automation?

GTINs or shop internal codes?

Mapping?

Filters?

Imputation?

Seasonal goods?

The same groups as before….

Thank you for your attention!

Comments or questions?