statistical analysis and discussion vinayak joshi ph.d. biomedical engineering pi: dr. joseph...
TRANSCRIPT
Statistical analysis and discussion
Vinayak JoshiPh.D. Biomedical EngineeringPI: Dr. Joseph Reinhardt Dr. Michael Abramoff
Overview of the process I work in ophthalmic/ eye fundus image
processing: Determination of tortuosity (curve/twist/kinks) index (Automated method)
Low tortuosity (TI=2.39) High tortuosity (TI=34.67)
Dataset evaluation Dataset of 25 images:
and so on……….till 25th image
Patient ID
Grader 1 Grader 2 Grader 3 Tortuosity Index
A 2 3 2 2.39
B 1 2 3 7.64
C 4 1 4 18.76
D 3 4 1 23.95
Problem 1 To validate the automated method or the
tortuosity index method by comparing it with the manual graders (gold standard).
Is the gold standard reliable? Or do the graders agree between themselves?
Solution: Fleiss’s Kappa (0< k < 1)
Patient ID
Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6
A 0 2 1 0 0 0
B 1 1 1 0 0 0
C 1 0 0 2 0 0
D 1 0 1 1 0 0
Till 25th rank
till 25th image
Problem 2How do we find the standard grading? Or average grading? Assuming that the graders agree mutually.
Patient ID
Grader 1
Grader 2
Grader 3
Tortuosity
Index
Average
Avg gradin
g
A 2 3 2 2.39 2.33 2
B 1 2 3 7.64 2 1
C 4 1 4 18.76 3 4
D 3 4 1 23.95 2.66 3
Problem 3 Likewise we developed a method for
automated measurement of tortuosity, previous researchers have also developed methods (10 methods) for the same purpose, but with some limitations.
Therefore, to prove our method to be superior among all, we need to prove that our method correlates with the manual grading with maximum correlation coefficient.
1 2 3 4 5 6 7 8 9 10 110
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Correlation coefficients for 11 methods, with the averaged manual grading
Proposed method: metric 11
How we can prove quantitatively/statistically that the maximum correlation obtained by our method (11) is significantly higher/better than other methods?
Probable solution We have currently estimated
measurements for 1 dataset of 25 images, for which we obtained correlation coefficients as,
DS 1 2 3 4 5 6 7 8 9 10 11
1 0.56 0.67 0.34 0.45 0.58 0.23 0.70 0.73 0.64 0.78 0.81
2 .53 .65 .23 .56 .65 .22 .71 .74 .66 .79 .79
3 .57 .67 .39 .45 .61 .31 .71 .75 .63 .69 .84
4 .61 .71 .4 .31 .49 .35 .76 .67 .59 .75 .83
5 .62 .56 .22 .52 .48 .18 .69 .65 .58 .77 .81
avg
stdv
Can we do ANOVA / BON-FERRONI and POST-HOC test now?NULL: all are equal, ALTERNATE: difference exists
THANK YOU