Breast tomosynthesis reduces Breast tomosynthesis reduces radiologist performance radiologist performance

variability compared to digital variability compared to digital mammographymammography

Andrew SmithAndrew Smith11, Elizabeth Rafferty, Elizabeth Rafferty22, Loren Niklason, Loren Niklason11

1 1 Hologic, Inc., Bedford MA, USAHologic, Inc., Bedford MA, USA2 2 Massachusetts General Hospital, Boston MA, USAMassachusetts General Hospital, Boston MA, USA

andrew.smith@hologic.comandrew.smith@hologic.com

Study GoalsStudy Goals

Measure radiologist variability of Measure radiologist variability of performance of 2D+3D compared performance of 2D+3D compared to 2D:to 2D:

–– Area under ROC curveArea under ROC curve–– Decision to recallDecision to recall

Tomosynthesis AcquisitionTomosynthesis Acquisition

XX--ray tube moves in an arc across the breastray tube moves in an arc across the breast11 projections over 1511 projections over 15°°Total Total tomotomo dose matched to 2D dose (FFDM)dose matched to 2D dose (FFDM)Reconstruction into 1 mm transverse slicesReconstruction into 1 mm transverse slices

Digital detector

Compression plate

Breast

X-ray tube

Reconstructed 1 mm planes

Data CollectionData CollectionStudy Study tomotomo in screening environment. 1000 in screening environment. 1000

patients. Diagnostic needed to enrich with cancers. patients. Diagnostic needed to enrich with cancers. Imaged using 2D (FFDM: Selenia), Imaged using 2D (FFDM: Selenia), Imaged using 3D (Tomosynthesis prototype)Imaged using 3D (Tomosynthesis prototype)Both views (CC, MLO), both breasts. Both views (CC, MLO), both breasts. Previous studies suggest both CC and MLO Previous studies suggest both CC and MLO tomotomo

needed for optimal performanceneeded for optimal performanceInstitution Review Board/Informed patient consentInstitution Review Board/Informed patient consent

Why 3D+2D, and not 3D alone?Why 3D+2D, and not 3D alone?Tomosynthesis (3D) will reduce tissue Tomosynthesis (3D) will reduce tissue

overlap compared to FFDM (2D), mostly overlap compared to FFDM (2D), mostly needed for massesneeded for masses

Having 2D exam speeds up search for Having 2D exam speeds up search for microcalcificationsmicrocalcifications

There will be a transition period when 2D is There will be a transition period when 2D is useful: comparison to priors, use of 2D as gold useful: comparison to priors, use of 2D as gold standard.standard.

System supports single breast compression System supports single breast compression acquisition of both 2D and 3Dacquisition of both 2D and 3D

Acquisition showing 3D Acquisition showing 3D followed by 2D examfollowed by 2D exam

Reader Study ProtocolReader Study Protocol~310 patients, includes ~310 patients, includes normalsnormals, recalls, , recalls,

benign lesions and 51 cancersbenign lesions and 51 cancers27 readers, radiologists with range of 27 readers, radiologists with range of

mammography experiencemammography experienceReaders had no prior experience with Readers had no prior experience with

tomosynthesistomosynthesisReaders were trained over 2 day sessionReaders were trained over 2 day session

Assessment MethodologyAssessment Methodology

Read images on dedicated mammography workstationRead images on dedicated mammography workstationRead and score the 2D:Read and score the 2D:

•• Initial BIRADS 0, 1, or 2Initial BIRADS 0, 1, or 2•• Forced BIRADS (1Forced BIRADS (1--5) 5) •• Probability of Malignancy (POM) from 0 to 100%Probability of Malignancy (POM) from 0 to 100%

Read 3D and score (2D+3D) using same methodsRead 3D and score (2D+3D) using same methods•• Initial BIRADS 0, 1, or 2Initial BIRADS 0, 1, or 2•• Forced BIRADS (1Forced BIRADS (1--5) 5) •• Probability of Malignancy (POM) from 0 to 100%Probability of Malignancy (POM) from 0 to 100%

Data AnalysisData Analysis

Compare 2D and (2D+3D) Compare 2D and (2D+3D) Compute ROC curve from Probability of Compute ROC curve from Probability of

MalignancyMalignancyROC analysis: MultiROC analysis: Multi--reader multireader multi--case case

DBM MRMC Beta 2 softwareDBM MRMC Beta 2 software

Examples of readersExamples of readers’’ ROC curvesROC curves

ROC Analysis: ROC Analysis: Area under ROC Curve (AUROC)Area under ROC Curve (AUROC)

AUROC of 3D+2D superior to 2D for AUROC of 3D+2D superior to 2D for every readerevery reader

Look at Look at radiologist variationradiologist variation, i.e. , i.e. standard deviation of AUROCstandard deviation of AUROC

Equality of Variance test for Equality of Variance test for statistical significancestatistical significance

Radiologist Variability with 2D and 2D+3D

2D+3D2D

All CasesAll Cases

0.0230.032Standard Deviation AUROC

0.8950.825Mean AUROC

2D+3D2D

AUROC 2D+3D variability 72% of 2D,AUROC 2D+3D variability 72% of 2D,variances equal p=0.11variances equal p=0.11

““AveragedAveraged”” ROC CurvesROC Curves

All cases All cases CalcsCalcs onlyonly No No calcscalcs

Cases involving CalcificationsCases involving Calcifications

0.0480.051Standard Deviation AUROC

0.8560.840Mean AUROC

2D+3D2D

AUROC 2D+3D variability 93% of 2D,AUROC 2D+3D variability 93% of 2D,variances equal p=0.73variances equal p=0.73

Cases without Cases without CalcsCalcs

0.0270.044Standard Deviation AUROC

0.9220.827Mean AUROC

2D+3D2D

AUROC 2D+3D variability 63% of 2D,AUROC 2D+3D variability 63% of 2D,variances equal p = 0.02variances equal p = 0.02

Decision to Recall,Decision to Recall,nonnon--cancer cases (BIRADS 0)cancer cases (BIRADS 0)

0.5300.413Kappa

78.7%70.9%Agreed on recall decision

2D+3D2D

Kappa differences statistically significantKappa differences statistically significant

Image ExamplesImage Examples

False Positive

Digital Mammogram Tomosynthesis Image

Digital Mammogram Tomosynthesis Image

Cancer

ConclusionsConclusions

Tomosynthesis previously shown to improve Tomosynthesis previously shown to improve radiologist performance: 3D+2D compared to radiologist performance: 3D+2D compared to 2D2D

Variability of AUROC reduced when using 3D, Variability of AUROC reduced when using 3D, for cases not involving calcifications, p = 0.02for cases not involving calcifications, p = 0.02

Variability unchanged for calc casesVariability unchanged for calc casesRadiologists are more accurate (AUROC) and Radiologists are more accurate (AUROC) and

more confident (Variances) with 3D+2D, more confident (Variances) with 3D+2D, compared to 2D for mass detectioncompared to 2D for mass detection

Radiologists agreed more often on decision to Radiologists agreed more often on decision to recall nonrecall non--cancer cancer cases with 2D+3D than 2Dcases with 2D+3D than 2D

More on the same subjectMore on the same subject……

Elizabeth Rafferty, MDElizabeth Rafferty, MDMassachusetts General Hospital, Boston MA, USAMassachusetts General Hospital, Boston MA, USA

InterInter--reader variability: Comparing breast reader variability: Comparing breast tomosynthesis combined with FFDM to FFDM alonetomosynthesis combined with FFDM to FFDM alone

Multi-center, retrospective study: Comparing breast tomosynthesis combined with FFDM to FFDM alone

Sunday, March 8, 14:00-15:30Satellite Symposium, Room F2

Thank youThank you

Images and data courtesy of:Images and data courtesy of:•• Massachusetts General Hospital, Boston MA USAMassachusetts General Hospital, Boston MA USA•• Netherlands Cancer Institute Netherlands Cancer Institute –– AntoniAntoni Van Leeuwenhoek Hospital, Amsterdam HollandVan Leeuwenhoek Hospital, Amsterdam Holland•• Centre de Centre de RadiologieRadiologie et et dd’’EchographieEchographie du du DocteurDocteur JoussierJoussier, Paris France, Paris France•• Dartmouth Hitchcock Medical Center, Lebanon NH USADartmouth Hitchcock Medical Center, Lebanon NH USA•• Magee WomenMagee Women’’s Hospital, Pittsburgh PA USAs Hospital, Pittsburgh PA USA•• University of Iowa Health Care, Iowa City IA USAUniversity of Iowa Health Care, Iowa City IA USA•• Yale University School of Medicine, New Haven CT USAYale University School of Medicine, New Haven CT USA