measuring diagnostic accuracy of using digital slides in routine histopathology and analyzing...

Measuring diagnostic accuracy of using digital slides in routine histopathology and analyzing sources of diagnostic errors

László FÓNYAD

1st. Dept. of Pathology and Experimental Cancer Research, Semmelweis University, Budapest, Hungary, 3DHISTECH Ltd., Budapest, Hungary

Measuring diagnostic accuracydigital slides in routine histopathology

The role of

digital slides

in pathology

Our questions

RecommendationsBackground Materials & methods Results

Designing a

retrospective,

comparative

study

Statistics can be

made to prove

anything –

even the truth

What should

the FDA

regulate?

Background

• Spreading of digital microscopes (DM) Worldwide is unquestionable.

• Reports have been published concerning the reliability of digital slides (DS) in

routine diagnostics showing great advances in the technical parameters of the

scanning process and in diagnostic confidence.

• The real revolution of DSs are still waited for. Today the most common fields of

using DSs are only research, education and as a paradox: quality controll.

• We wanted to estimate the major causes of dislike and/or dissatisfaction that could

explain the mistrust in DS therefore interfere the real breakthrough of this techique in

the routine work.


Background

Our questions were:

• Can we define a list of samples according to the origin and type of stain etc.

where the DM is sufficient to use in routine practice and define those where it

is not recommended?

• Is it possible to estimate the type of errors resulting in misdiagnosed cases?

• Does the pathologists’ interpretative skills and experience effect the diagnostic

results using DM and how important these factors are comparing to the actual

quality of the DS and working conditions?


Materials & methods

Materials:

•Scanners and softwares were provided by the 3DHISTECH Ltd., Budapest

•280 cases were enrolled to the study

•1771 slides were scanned

•994 H&E, 70 Giemsa, 174 other special stains (mainly PAS, Prussian-blue, picrosyrius, Masson's trichrome), 553 immunohistochemistry slides. No smears or cytology samples were scanned.

•1530 were evaluated by the pathologists for digital diagnose

Table 1., Enrolled cases according to the localisation

skin 49

bone marrow 10

breast 26

upper GI-tract 37

soft tissue 21

liver 26

lymphnode 18

thyroid gland 15

lung 22

large bowel 27

kidney 29


Materials & methods

Method:

•7 pathologists

•Pathologista A, B and D received cases, specific to their field. Pathologist C, E, F, G received non-field specific cases too.

•Initially, only those slides that were available for the first assessment (mostly HE) were uploaded to the digital database.

•A Clinical Researc Form was filled out.

•The diagnostic concordance and the reasons related DS to diagnostic uncertainity were analyzed.

•The incoherent cases were graded and 4 types of diagnostic errors were defined.

A. Scan quality

1-Unacceptable

2-Poor

3-Adequate

4-Good

5-Excellent

The reason of dissatisfaction

Important areas of the slide are out of focus (y/n)

Incomplete scan (y/n)

The color fidelity is poor (y/n)

Other (free text)

B: Diagnostic confidence

1-Uncertain

2-Likely

3-Confident

The reason of uncertainty is due to:

Case complexity (y/n)

Poor image quality (y/n)

Types of diagnostic error

Type I. non relevant incoherence – uncertainty recorded.

Type II. non relevant incoherence – uncertainty not recorded.

Type III. relevant incoherence – uncertainty recorded.

Type IV. relevant incoherence – uncertainty not recorded.


Results

Technical results:

•Scantime ratio: 1,215 min/cm2

•Average quality of the 1530 slides was 4,45/5.

•At 34 slides the reason of dissatisfaction was that “important

areas of the slide were out of focus”. Twice the scan was

considered incomplete and 10 times the color fidelity was

rated poor.


Results

Diagnostic results

diagnostic confidence (1-3)

uncertainty due to poore image quality

concordant

discordant reassesed incoherent non-rel - recorded

non-rel - missed

rel - recorded

relevant - missed

2,74 4,64% 78,57% 21,43% 9,64% 11,79% 1,07% 2,50% 5,71% 2,50%


Results

Diagnostic results

Can we define a list of samples according to the origin where the DM is sufficient to use in routine

practice and define those where it is not recommended?


Results

Diagnostic results

Does the pathologists’ interpretative skills and experience effect the diagnostic

results using DM?

Results excluding non-field specific cases:

number of cases

concordant discordant reassesed incoherent non rel - recorded

non rel - missed

rel - recorded

rel - missed

% % % % % % % %

280 78,57% 21,43% 9,64% 11,79% 1,07% 2,50% 5,71% 2,50%

172 83,14% 16,86% 8,72% 8,14% 0,58% 0,00% 5,81% 1,74%


Recommendations

1. Based on others and our results we think that the level of diagnostic confidence using digital

slides are acceptable.

2. Reasons responsible for diagnostic errors are mostly personal and reflects the competency of

the examiner.

3. Technical reasons, potentially responsible for errors (poor color fidelity, blurred image) are

detectable by the examiner and correction of it could be initiated. (rescan, recut, restain etc.)

4. However strict regulations required for the scanning process inserted to the

prediagnostic phase

• Safety of sample recognition: how do we prevent data loss because of incomplete scan.

• Proper glass slide handling: provide accurate slide ID recognition.

• Minimalize the chance of breaking glass slides during scanning process.


Measuring diagnostic accuracy of using digital slides in routine histopathology and analyzing sources of diagnostic errors

László FÓ[email protected]

1st. Dept. of Pathology and Experimental Cancer Research, Semmelweis University, Budapest,, Hungary

Thank You for Your attention!