statistical analysis and discussion vinayak joshi ph.d. biomedical engineering pi: dr. joseph...

Statistical analysis and discussion

Vinayak JoshiPh.D. Biomedical EngineeringPI: Dr. Joseph Reinhardt Dr. Michael Abramoff

Overview of the process I work in ophthalmic/ eye fundus image

processing: Determination of tortuosity (curve/twist/kinks) index (Automated method)

Low tortuosity (TI=2.39) High tortuosity (TI=34.67)

Dataset evaluation Dataset of 25 images:

and so on……….till 25th image

Patient ID

Grader 1 Grader 2 Grader 3 Tortuosity Index

A 2 3 2 2.39

B 1 2 3 7.64

C 4 1 4 18.76

D 3 4 1 23.95

Problem 1 To validate the automated method or the

tortuosity index method by comparing it with the manual graders (gold standard).

Is the gold standard reliable? Or do the graders agree between themselves?

Solution: Fleiss’s Kappa (0< k < 1)

Patient ID

Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6

A 0 2 1 0 0 0

B 1 1 1 0 0 0

C 1 0 0 2 0 0

D 1 0 1 1 0 0

Till 25th rank

till 25th image

Problem 2How do we find the standard grading? Or average grading? Assuming that the graders agree mutually.

Patient ID

Grader 1

Grader 2

Grader 3

Tortuosity

Index

Average

Avg gradin

g

A 2 3 2 2.39 2.33 2

B 1 2 3 7.64 2 1

C 4 1 4 18.76 3 4

D 3 4 1 23.95 2.66 3

Problem 3 Likewise we developed a method for

automated measurement of tortuosity, previous researchers have also developed methods (10 methods) for the same purpose, but with some limitations.

Therefore, to prove our method to be superior among all, we need to prove that our method correlates with the manual grading with maximum correlation coefficient.

1 2 3 4 5 6 7 8 9 10 110

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Correlation coefficients for 11 methods, with the averaged manual grading

Proposed method: metric 11

How we can prove quantitatively/statistically that the maximum correlation obtained by our method (11) is significantly higher/better than other methods?

Probable solution We have currently estimated

measurements for 1 dataset of 25 images, for which we obtained correlation coefficients as,

DS 1 2 3 4 5 6 7 8 9 10 11

1 0.56 0.67 0.34 0.45 0.58 0.23 0.70 0.73 0.64 0.78 0.81

2 .53 .65 .23 .56 .65 .22 .71 .74 .66 .79 .79

3 .57 .67 .39 .45 .61 .31 .71 .75 .63 .69 .84

4 .61 .71 .4 .31 .49 .35 .76 .67 .59 .75 .83

5 .62 .56 .22 .52 .48 .18 .69 .65 .58 .77 .81

avg

stdv

Can we do ANOVA / BON-FERRONI and POST-HOC test now?NULL: all are equal, ALTERNATE: difference exists

THANK YOU