Download - Local and Global Scores in Selective Editing
![Page 1: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/1.jpg)
1
Local and Global Scores in Selective Editing
Dan Hedlin
Statistics Sweden
![Page 2: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/2.jpg)
2
Local score
• Common local (item) score for item j in record k:
• wk design weight
• predicted value
• zkj reported value
j standardisation measure
jkjkjkkj zyw ~~
kjy~
![Page 3: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/3.jpg)
3
Global score
• What function of the local scores to form a global (unit) score?
• The same number of items in all records
• p items, j = 1, 2, … p
• Let a local score be denoted by kj
• … and a global score by kg γ
![Page 4: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/4.jpg)
4
Common global score functions
In the editing literature:
• Sum function:
• Euclidean score:
• Max function: kjj
max
p
jkj
1
2
p
jkj
1
![Page 5: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/5.jpg)
5
• Farwell (2004): ”Not only does the Euclidean score perform well with a large number of key items, it appears to perform at least as well as the maximum score for small numbers of items.”
![Page 6: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/6.jpg)
6
Unified by…
• Minkowski’s distance
• Sum function if = 1
• Euclidean = 2
• Maximum function if infinity
1
1
;
p
jkjkg γ
1
![Page 7: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/7.jpg)
7
• NB extreme choices are sum and max
• Infinite number of choices in between = 20 will suffice for maximum unless
local scores in the same record are of similar size
![Page 8: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/8.jpg)
8
Global score as a distance
• The axioms of a distance are sensible properties such as being non-negative
• Also, the triangle inequality
• Can show that a global score function that does not satisfy the triangle inequality yields inconsistencies
lklk ggg γγγγ
![Page 9: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/9.jpg)
9
• Hence a global score function should be a distance
• Minkowski’s distance appears to be adequate for practical purposes
• Minkowski’s distance does not satisfy the triangle inequality if < 1
• Hence it is not a distance for < 1
![Page 10: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/10.jpg)
10
Parametrised by
• Advantages: unified global score simplifies presentation and software implementation
• Also gives structure: orders the feasible choices…from smallest: = 1…to largest: infinity
![Page 11: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/11.jpg)
11
• Turning to geometry…
![Page 12: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/12.jpg)
12
Sum function = City block distance
p = 3, ie three items
![Page 13: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/13.jpg)
13
Euclidean distance
![Page 14: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/14.jpg)
14
Supremum (maximum, Chebyshev’s) distance
![Page 15: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/15.jpg)
15
Imagine questionnaires with three items
1k
Record k2k
3k Euclidean distance
![Page 16: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/16.jpg)
16
![Page 17: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/17.jpg)
17
The Euclidean function, two items
A sphere in 3DThreshold
Threshold
![Page 18: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/18.jpg)
18
The max function
A cube in 3D Same threshold
![Page 19: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/19.jpg)
19
The sum function
An octahedron in 3D
![Page 20: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/20.jpg)
20
![Page 21: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/21.jpg)
21
• The sum function will always give more to edit than any other choice, with the same threshold
![Page 22: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/22.jpg)
22
Three editing situations
1. Large errors remain in data, such as unit errors
2. No large errors, but may be bias due to many small errors in the same direction
3. Little bias, but may be many errors
![Page 23: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/23.jpg)
23
Can show that if…1. Situation 32. Variance of error is
3. Local score is
• Then the Euclidean global score will minimise the sum of the variances of the remaining error in estimates of the total
2~kjkjkj zyVar
jkjkjkkj zyw ~~
![Page 24: Local and Global Scores in Selective Editing](https://reader030.vdocuments.net/reader030/viewer/2022033105/568135bd550346895d9d2282/html5/thumbnails/24.jpg)
24
Summary
• Minkowski’s distance unifies many reasonable global score functions
• Scaled by one parameter• The sum and the maximum functions are
the two extreme choices• The Euclidean unit score function is a good
choice under certain conditions