New validation method and IT system in DK

Søren BurmanNordic meeting 2014, Tórshavn

2

The validation system

Where to make the cut…

Manage and edit cases

Future plans

Outline

3

The automatic validation system consists of three procedures.

1. For each enterprise, the data is tested against its historical reports on the service item level

A score is applied to all observations based on:

1. How large is the given observation compared to earlier values for the given cell.

2. How large is the normal variation for the service item in question.

3. How large is the observation in absolute terms.

The validation system

4

2. For each enterprise, the data is tested against its historical reports on the total flow

This is done in order to capture missing trade Similar scoring system as in procedure 1, but with a lower

”impact” Captures enterprises that suddenly report zero trade

The validation system

5

3. Check for use of new service items If a new service item code is used that has never been

used before, it cannot be tested against its historical reports

Captures errors where enterprises use a wrong code, but maintain the same level of trade

New reporters will not be treated in the validation system until six reports are approved

All new reporters must be manually approved until six approved reports are available

The validation system

6

Less than six approved reports

The validation system

Automatic validation

system

Procedure 1

Procedure 2

Procedure 3

Non validated reports

Possible error list – sorted

by value

Possible error list – sorted

by score

Manual approval list

Cut off

What is considered

”wrong” enough

7

Not all deviations are errors Score gives an indication of severity Below an estimated view of the hit-rate for the year 2012

Where to make the cut…

Score Number of lines Hits Share

100+ 115 84 73.04%30+ 207 57 27.54%20+ 154 29 18.83%15+ 138 27 19.57%10+ 257 28 10.89%7+ 254 18 7.09%5+ 262 23 8.78%2+ 673 39 5.79%Below 2 1492 56 3.75%

8

A case is defined as a report from an enterprise Many of the suspicious lines are linked to the same

enterprise, so the actual number of cases is lower

How many cases do we “usually” process

Cut of from above, knowing that many suspicious lines are not errors

Where to make the cut…

9

All cases having at least one line with a score higher than 15 are handled by the case reviewers 30 to 40 cases pr. month ~ 10 pct. of monthly reports

On top of the that: ~10 cases from procedure 3 Ad hoc chosen levels:

- Above 10.000.000 DKK OR- Above 2.500.000 and more than half of the trade

Roughly put: 50 pct. of the cases are hits.

Where to make the cut…

10

All cases are stored in one system Grouped by enterprise Sorted by highest score across the reports

Different filtering options available Automatically generated cases >< Manual cases Suspicious lines >< New service codes New reporters Blocked or non-blocked cases Case worker identity

Manage and edit cases

11

Manage and edit cases

12

Double clicking on a case brings up the overview of the enterprise Errors are marked with red Yellow indicates cells that are changed Purple indicates that the reports are blocked

Manage and edit cases

13

Edit data by double clicking on a cell

Enterprise and period

Data for given cell

Development over time

History for the given cell

Manage and edit cases

14

Future plans

Simplify and enhance the filtering options in the overview of the cases

Add more automatic validation procedures Merchanting The geographical level (especially country codes 4A and 7Z) Construction Insurance

15

How is ITS data validated in your country?

Validating trade in services