The PET/CT Working Group: 2013-2014

.

CT Segmentation Challenge Informatics IssuesMulti-site algorithm comparison

Task: CT-based lung nodule segmentationEvaluate algorithm performance

Bias, repeatability of volumesOverlap measuresUnderstand sources of variability

• 52 nodules from 5 collections hosted in The Cancer Imaging Archive (TCIA)

• LIDC (10)• RIDER (10)• CUMC Phantom (12)• Stanford (10) • Moffitt (10)

Data

Participants and Algorithms

• CUMC: marker-controlled watershed and geometric active contours

• Moffitt Cancer Center: multiple seed points with region growing. Ensemble segmentation obtained from the multiple grown regions.

• Stanford University: 2.5 dimension region growing using adaptive thresholds initialized with statistics from a “seed circle” on a representative portion of the tumor

Future Plans

Informatics

PET Segmentation Challenge

• Phase II: Hardware phantom scanned at 2+ sites (UI, UW)

• NEMA IEC Body Phantom Set™• Model PET/IEC-BODY/P • Four Image Sets per Site• Generate accurate volumetric

segmentations of the objects in the phantom scans

• Calculate the following indices for each of the objects: VOI volume, Max, PEAK & AVERAGE Concentration, Metabolic Tumor Volume

•Phase III: PET challenge with clinical data (H&N)

Four phase challenge starting with software phantom (DRO), hardware phantom scanned at multiple sites, segmenting clinical data and correlating PET with outcomes

Phase I: DRO

Future Plans

Results

• Catalog of CT segmentation tools• Feature extraction project: Assess impact of segmentations on

features (shape, texture, intensity) implemented at different QIN sites

• Created converters for a range of data formats• Used TaCTICS to compute metrics• Agreed to use DICOM-SEG or DICOM-RT for future

segmentation challenges• Exploring use of NCIPHUB for future challenges

• Digital Reference Object: Generated by UW/QIBA

• 7 QIN sites participated• UW, Moffitt, Iowa, Stanford,

Pittsburgh, CUMC, MSKCC• Software packages used included PMOD, Mirada

Medical RTx, OSF tool, RT_Image, CuFusion, 3D Slicer

• After some effort, all sites were able to calculate the DRO SUV metrics correctly

Nodules volume by collection. Most nodules in the LIDC and phantom collection were small while others had a wide range of sizes

All pairwise dice coefficients (all runs, all algorithms by nodule) by collection indicates shows better agreement between algorithms on the phantom nodules (CUMC) than on clinical data (ANOVA, p<0.05)

Intra-algorithm agreement was much higher than inter-algorithm agreement (p <0.05)

Exploring causes of variability

Some nodules (e.g., Lg) have high variability (typically heterogeneous)

Estimated volume by algorithm for highlighted

nodule

Bias in estimating volume (phantom data, 12 nodules)

Each site submitted 3-4 segmentations for each of the 52 nodules allowing the assessment of intra- and inter-algorithm agreement

Bias (estimated-true volume) for CUMC-phantom nodules shows a difference between algorithms (ANOVA with blocking, p <<0.05) Reproducibility of algorithms

Examining bias for large and small nodules

Statistically significant difference in intra-

algorithm Dice coefficients

Volume by lesion as measured by repeat runs of

algorithms

Dice coefficient (all algorithms, all runs) of

nodules in Stanford collection (ordered by volume left to right)