1 some metrological aspects of ordinal quality data treatment *emil bashkansky tamar gadrich ort...

1

*Emil BashkanskyTamar Gadrich

ORT Braude College of

Engineering, Israel

ENBIS-11 Coimbra, Portugal, September 2011, 11:50 – 13:20

2

3

4

5

Examples of ordinal scale usageExamples of ordinal scale usage

DAILY LIFEDAILY LIFE MEDICINEMEDICINEQUALITY QUALITY

MANAGEMENTMANAGEMENT EngineeringEngineering

• Sports results:

a win, tie, loss• Voting results:

pro, against,

abstain• Academic

ranks• …

• Rankin score

(RS) - level of

disability

following a

stroke • Side effect

severity• …

• Quality level

estimation and

sorting• Customer

satisfaction

surveys • Ratings of

wine colour,

aroma and

taste • FMECA• …

• The Mohs

scale of

mineral

hardness• Dry-chemistry

dipsticks (e.g.,

urine test)• The Beaufort

wind force

scale• ....

6

* ISO/IEC Guide 99: International vocabulary of metrology —

“Basic and general concepts and associated terms (VIM)”

7

Classic continual:

The probability density function pdf (Y/X) of receiving result Y, given the true value of the measurand X , in it's simplest form:

pdf (Y/X) = Normal (X+bias, ) Ordinal:

The conditional probabilities that an object will be classified as level j, given that its actual/true level is i .

2

/ (1 , )j i i j mP

/1

1m

j ij

P

8

1 0 ... 0

0 1 ... 0

0 0 ... 1

P I

9

Factual

Actu

al + -+-

1-α α

1- ββ

10

(4)1 0 0.5 0.5 0 1 1 0

(1) (2) (3)0 1 0.5 0.5 1 0 1 0

P P P P

11

The likelihood that a measured level j is

received, whereas the true level is i

1

//

/

m

i

p Pi j iQ i j

p Pi j i

1

//

/

m

i

Pj iQi j

Pj i

/1

1m

i ji

Q

12

1

1

1

11

1

Q̂

13

/E P Ii j i jj i

1 1( )

2

2/

m m

i jP I

Am

i jj i

(0 1)A

2 20.5 ( )A

Error matrix:

A

15

16

Blair & Lacy (2000)

)4

1(

)1(1

1

m

FFVAR

m

k kk

0 1VAR

17

1

1/ /(1 )

4 1 ,1

[ ]m

kk i k iF F

VAR i j mmi

1

2

4 (1 )

4 (1 )

VAR

VAR

VAR

/ /1

k

k i j ij

F P

- the expected cumulative frequency of data/items classified up to the k-th category, given that its actual/true level is i

18

1

2/ .

1 1 1

1

1 4

K K K

i i i k i ki k i

totalVAR p VAR p F FK

. /1

K

k i k ii

F p F

- the expected cumulative frequency of items belonging up to the k-th category after measurement

ORDANOVA: DECOMPOSITION OF TOTAL DISPERSION AFTER MEASUREMENT/CLASSIFICATION

19

20

)(iba

- conditional joint probability of sorting the measured object to the a-th level by the first MS (called A), and the b-th level by the second MS (called B), given the actual/true category i

111

)(

m

b

m

a

iba

21

A & B MSs classification matrices

( ) ( )

1. /

m

ba a i ab

i iP

( ) ( )

1. /

m

ab b i ab

i iP

22

( )

1

m

i

piab abi

pi - the probability that an object being measured

relates to category i, ( )1ip

ab - the joint probability of sorting the item as a by the first measurement system (A) and b by the second measurement system (B).

23

MODIFIED KAPPA MEASURE OF AGREEMENT

1

1( )

actual agreement agreement by chance

agreement by chance

agreement by chance

P P

P

P m

m 2 3 4 5 6 7 8 9 10 20 100

0

0.2

5

0.3

3

0.3

8 0.4 0.42 0.43 0.44 0.44

0.4

7 0.5

When a half of all items are correctly classified:

24

1

1

11

mi

kki k

m

m

25

1

1

11

m

kkk

m

m

1

mi

ii

p

26

B B

A

ABn – observed number of

units in which the feature was

found by both A and by B

ABn – observed number of

units in which the feature was

found by A but not by B

A

ABn – observed number of

units in which the feature was

not found by A but was found

by B

ABn – number of units in

which the feature was missed by

both A and B is unknown!

Binary case example

2

1ABABn n n

27

QUALITY CATEGORY

SOLUBLE SOLIDS

CONTENT(SSC)

TITRATABLE ACIDITY

(TA)PH

TOTALSUGARS

MASS FRACTIONS

SKIN COLOR("A" VALUE)

FLESH FIRMNESS

MASS

  (%) (%)   (G/KG)   (N) (G)HIGH (TYPE 1) 11.2-

15.30.51-1.01

3.6-3.9 80-110 5-29 7-28 >105

MEDIUM (TYPE 2)

11.2-14.8

0.66-1.16

3.5-3.8 50-80 13-30 25-60 78-115

LOW (TYPE 3) <12 >1.1 <3.6 <50 >25 >55 <85

Typical relation between quality level and commonly used chemical/physical features for yellow-flesh nectarines

28

i 1 2 3

ip 0.53 0.27 0.2

i

0.84 0.03 0.02

0.02 0.04 0.02

0.02 0.01 0.00

0.04 0.02 0.01

0.03 0.76 0.06

0.01 0.07 0.00

0.01 0.02 0.02

0.03 0.02 0.10

0.03 0.10 0.67

i 0.82

0.70

0.55

0.89 0.08 0.03

ˆ 0.07 0.85 0.08

0.05 0.15 0.80AP

0.88 0.08 0.04ˆ 0.08 0.85 0.07

0.07 0.14 0.79BP

( ) 0.792A ( ) 0.781B

weighted total kappa equals 0.734

Ternary scale example(fruit quality classification)

0.458 0.0253 0.0173

0.0247 0.2304 0.0468

0.0193 0.0442 0.134

33

Let's consider arbitrary ordinal scale with m categories and suppose, that n repeated measurements of the same object were performed resulting in vector:

(n=n1+n2+…+nm)

},...,{21 mnnnn

34

/ /11

1 1

! !exp ln

! !

jm m

n

i j i j j im mjj

j jj j

n nL P n P

n n

The maximum likelihood estimation must be made in favor of such, most plausible i , that maximizes the scalar product:

/1

[ ln( )]j i

m

jj

n P n lni

1 2{ln( ), ln( )...ln( )}/ / /ln P P Pi i m ii

35

(1)0.89 0.08 0.03

ˆ 0.07 0.85 0.08

0.06 0.14 0.80

P

0.254

0.2774

0.4328

pVAR

36

50.9997 0.0002 9.9 10

(10 )0.0003 0.9971 0.0026

53.1 10 0.0014 0.9986

ˆ

P 10

0.0008

0.0058

0.0028P

VAR

(1)0.89 0.08 0.03

ˆ 0.07 0.85 0.08

0.06 0.14 0.80

P0.254

0.2774

0.4328

pVAR

38

1.On an ordinal measurement scale the essential for evaluating the error, repeatability and uncertainty of the measurement result base knowledge must be the classification/measurement matrix. Given this matrix, authors introduced a way to calculate the classification/measurement system’s accuracy, precision (repeatability & reproducibility) and uncertainty matrix.

2.In order to estimate comparability and equivalence between measurement results received on an ordinal scale basis, the modified kappa measure is suggested. Three of the most suitable usages of the measure were thoroughly analyzed. The advantage of the proposed measure vs. the traditional one lies in the fact that the former follows the superposition principle: the total measure equals the weighted sum of partial measures for every ordinal category.

3.As it is well known, repeated measurements may improve the quality of the measurement result. When decisions are ML based, one can find how many repetitions are necessary in order to achieve the desired accuracy level using the algorithm suggested by the authors,.

40

E-mail: ebashkan@braude.ac.il

41