thesis advisor: dr. mandayam committee: dr. kadlowec and dr. polikar

Automated evaluation of Automated evaluation of radiodensities in a digitized radiodensities in a digitized

mammogram database using local mammogram database using local contrast estimationcontrast estimation

Automated evaluation of Automated evaluation of radiodensities in a digitized radiodensities in a digitized

mammogram database using local mammogram database using local contrast estimationcontrast estimationThesis Advisor: Dr. Mandayam

Committee: Dr. Kadlowec and Dr. Polikar

Friday, July 23, 2004

OutlineOutline

• Introduction

• Objectives of the Thesis

• Previous Work

• Approach

• Results

• Conclusions

Cancer Related Deaths in the U.S. (women)

Urinary3.22%

Skin1.33% Soft Tissue

0.70%

Bones0.22%

Respiratory System25.98%

Digestive System22.80%

Breast14.71%

Genital9.90%

Oral/Phalanx0.89%

Other7.61%

Leukemia3.62%

Mulitple Myeloma2.03%

Lymphoma4.36%

Eye0.04%

Endocrine0.44%

Brain2.14%

IntroductionIntroduction

Breast CancerBreast CancerNew Cases of Cancer in the

U.S. (women)

Breast32.07%

Genital12.70%

Digestive System18.23%

Skin4.02%

Soft Tissue0.58%

Respiratory System12.69%

Bones0.17%

Oral/Phalanx1.44%

Other2.46%

Leukemia1.93%

Mulitple Myeloma1.03%

Urinary4.31%

Eye0.17%

Brain1.23%

Endocrine2.61%

Lymphoma4.36%

Survival RatesSurvival Rates

0 100%

I 98%

IIA 88%

IIB 76%

IIIA 56%

IIIB 49%

IV 16%

Each stage designates the size of the tumor how much it has spread.

Stage 0 Cancer:

Lobular Carcinoma in Situ (LCIS)

Ductal Carcinoma in Situ (DCIS)

20% of all diagnosed cancers

Mammography ProcedureMammography Procedure

Compression Plate

Compression Plate

Film Holder

Pectoral Muscle

Film Holder

MLOView

CCView

Risk Factor High-Risk Group Low-Risk Group Relative risk

Age Old Young > 4.0

Country of birth North America, Northern Europe

Asia, Africa > 4.0

Socioeconomic status High Low 2.0 – 4.0

Marital Status Never married Ever married 1.1 – 1.9

Place of residence Urban Rural 1.1-1.9

Place of residence Northern US Southern US 1.1-1.9

Race ≥ 45 years < 40 years

WhiteBlack

BlackWhite

1.1-1.91.1-1.9

Nulliparity Yes No 1.1-1.9

Age at first full-term pregnancy ≥ 30 years < 20 years 2.0-4.0

Age at menopause Late Early 1.1-1.9

Weight, postmenopausal women Heavy Thin 1.1-1.9

Any first-degree relative with history of breast cancer

Yes No 2.0-4.0

Mother and sister with history of breast cancer

Yes No > 4.0

Mammographic parenchymal patterns

Dysplastic Normal 4.0-6.0

Risk FactorsRisk Factors

Breast densityBreast density

Chest wall

Pectoralis muscles

Lobules

Nipple surface

Areola

Duct

Fatty tissue

Skin

Radiodense

Tissue

Radiolucent

Tissue

RadiodensityRadiodensity

Chest wall

Pectoralis muscles

Lobules

Nipple surface

Areola

Duct

Fatty tissue

Skin

Radiodense

Tissue

Radiolucent

Tissue

Mammographic DensityMammographic Density

“……..women who had a breast density of 75% or greater had an almost fivefold increased risk of breast cancer…………”

– Byrne, C, et. al. “Mammographic features and breast cancer risk: effects with time, age, and menopause status,” Journal of the National Cancer Institute, Vol. 87, pp.1622-1629, 1995.

Genetic HeritabilityGenetic Heritability“Women with extensive dense breast tissue visible on

mammogram have a risk of breast cancer that is 1.8 to 6.0 times that of women of the same age with little or no density.”

“…………….. the percentage of dense tissue on mammography at a given age has high heritability. Because mammographic density is associated with an increased risk of breast cancer, finding the genes responsible for this phenotype could be important for understanding the causes of the disease.”

– Boyd, N.F., et al, “Heritability of mammographic density, a risk factor for breast cancer,” New England Journal of Medicine, Volume 347(12), September 19, 2002, pp. 886-894.

IssuesIssues

• Current methods are still slow and subjective.

• Variability still exists between radiologists.

• Automated algorithm for fast and objective estimation.

• Rowan University.

Objectives of this ThesisObjectives of this Thesis

• Investigate the use of textures for the segmentation of radiodense tissue in a digitized mammogram.

• Create an automated algorithm that is able to consistently evaluate digitized mammograms throughout several databases.

• Compare the results of the algorithm to a established manual methods, the “Toronto” method as well as previous methods created at Rowan University.

TexturesTextures

Region of Mammogram

Texture Description Method

Feature 1

Feature 2

Feature 3

Feature …

Un-supervised Clustering Method

Mammogram

Total Bank of Features

Classified Mammogram

f1 f2 f3 … fn

f1(1,1) f1(1,2) f1(1,3) …f1(1,j)

f1(2,1) f1(2,2) f1(2,3) …f1(2,j)

f1(3,1) f1(3,2) f1(3,3) …f1(3,j)

…f1(i,1) …f1(i,2) …f1(i,3) …f1(i,j) …

Automated AlgorithmAutomated Algorithm

Database 1

Estimated percentages for database 1

Automated Process

Database 2

Estimated percentages for database 2

Validation PercentagesCompare Compare

Performance on database 1

Performance on database 2

PERFORMANCES ‘”SHOULD” BE

SIMILAR

Previous WorkPrevious Work

• Wolfe’s classification.

• “Toronto” method.

• Automated techniques.– “Main goal of research conducted at

Rowan University”

Wolfe’s ClassificationWolfe’s Classification

• N1: The breast is comprised entirely of fat.

• P1: The breast has up to 25% nodular densities.

• P2: The breast has over 25% nodular mammographic densities.

• DY: The breast contains extensive regions of homogeneous mammographic densities.

““Toronto” MethodToronto” Method

Display Results

33.3% RD

66.6% RL

Load Image into Computer

Set Boundary Threshold

1 4096

Set TissueThreshold

1 4096

Count pixels inregions

1 4096

Display Results

33.3% RD

66.6% RL

Load Image into Computer

Set Boundary Threshold

1 4096

Set TissueThreshold

1 4096

Count pixels inregions

1 4096

AutomatedAutomated

Proponents Approach Advantages Disadvantages

Lou and Fan [35] Adaptive fuzzy K-means technique to classify pixels as radiodense.

7.98 % error among 81 mammogram images.

18 seconds process time per image.

Zou et al. [36,37] Rule based histogram classifier

Maximum difference 20% from expert analysis.

No objective method for validation.

Bovis and Singh [38]

Classification using texture analysis.

91 % correct classification. Relies on knowledge of the region to be segmented.

Classifier is based on simplistic measures of texture.

Saha, Udupa, et al. [39]

Scale-based fuzzy connectedness

models

Estimates correlate strongly with analysis by radiologist.

Does not automatically exclude pectoral muscle.

Neyhart et al. [40]Eckert et al.

[41]

Constrained Neyman-Pearson decision

functionw/wo

Compression Adjustment

Automated technique Performance fit to database tested with. Weak inter-

dataset performance.

Neyman-Pearson ClassifierNeyman-Pearson Classifier

Distribution 1(Radiolucent)

Distribution 2(Radiodense)

12, 21=2

2

12

212

2

NPT

1 2

TNP1 2

TNP

Gray-level intensity

Num

ber

of P

ixel

s

1 and 2 are means of distributions, 2 is local variance of image

• Varies threshold based on the variance of image from pure Bayesian to 2

• Can compensate for brightness of image and classify image radiodensity

• Determine from training data set

Constrained Neyman-Pearson Constrained Neyman-Pearson ClassifierClassifier

2212

221

CNPT

Spatially Varying CNPSpatially Varying CNP

Compression Plate

Film Holder

CCView

More StressHere

Less StressHere

More DensityHere

Less DensityHere

Compression CompensationCompression Compensation

Multiple lowpass filtering operations

IssuesIssues

• Most of these algorithms are not fully automated.

• Performance is evaluated in just one type of database.

ApproachApproach

• Texture and image processing methods investigated.

• Local Contrast Estimation algorithm.

• Investigation of previous methods created at Rowan University.

TexturesTextures

• If radiodense and radiolucent tissue exhibit characteristics that are different from each other, texture…

• Evaluation of 3 different ‘types’ of methods.

Texture description methods

VarianceVariance

10 images:

5 FCC

5 Harvard

Database Evaluation

Individual Variance Imaging

Intra-image characteristics

evaluation

Inter-image and cross-database

statistic evaluation

Variability of texture characteristics

Variance ImagingVariance Imaging

Regional Variance Imaging

Histogram

Histogram

Gabor FilteringGabor Filtering

})([2exp{})([2exp{),( 220

22220

22 vuuvuuvuH yxyx )2cos(2

1exp

2

1),( 02

2

2

2

xyx

yxhyxyx

Spatial Domain Frequency Domain

Gabor FilteringGabor Filtering

50 regionalsamples forradiodensetissue

50 frequencyprofiles forradiodensetissue

2-Dimensional FFT

2-Dimensional FFT

50 regionalsamples forradiolucenttissue

50 frequencyprofiles forradiolucenttissue

averagedfrequencyprofile for radiodensetissue

averagedfrequencyprofile for radiolucenttissue

highestregion ofdifference

Co-occurrenceCo-occurrence

|}),(,),(,||,0:)],(),,{[(|),(0 bnmfalkfdnlmkDnmlkbaP

0 0 1 10 0 1 10 2 2 22 2 3 3

4 2 1 02 4 0 01 0 6 10 0 1 2

),(0 baP

Co-occurrenceCo-occurrence

ba

d baP,

2, ),(

)),((log),( ,,

2, baPbaP dba

d

ba

dk baPba

,, ),(||

baba

k

d

ba

baP

,,

,

||

),(

Energy

Moments

Entropy

Inverse Moments

Law’s Texture Energy MeasuresLaw’s Texture Energy Measures

• Spatial filters based on three simple vectors:– Averaging L = (1,2,1)– Edges E = (-1,0,1)– Spots S = (-1,2,1)

• These 3 vectors can be combined to make 25 separate spatial filters.

Law’s Texture FilterLaw’s Texture FilterL5 = [ 1 4 6 4 1 ] E5 = [-1 -2 0 2 1 ] S5 = [-1 0 2 0 -1 ] W5 = [-1 2 0 -2 1 ] R5 = [ 1 -4 6 -4 1 ]

1 4 6 4 1 4 16 24 16 4 6 24 36 24 6 4 16 24 16 4

1 4 6 4 1

1

4

6

4

1

[ 1 4 6 4 1 ]

3 Simple Vectors

All convolution pairs

5 Vectors

All column by row multiplication pairs

25 matrices (filters)Filtering +energy

measure + addition of Complements

Set of 15 features

Image

Averaging L = (1,2,1)Edges E = (-1,0,1)Spots S = (-1,2,1)

[1 2 1]*[1 2 1] = [1 4 6 4 1]

x =

25 filtersFiltering Result Energy measuring function

Complements are summed

15 sets of features

ClusteringClustering

• Variance Imaging = 1 feature.

• Co-occurrence = 4 features.

• Law’s Energy Measure = 15 features.

Supervised Learning Techniques are not viable because of the vast variation texture characteristics!!!

k-meansk-meansbegin initialize n, c, µ1, µ2 … µc

do classify n samples according to nearest µi

recompute µi

until no change in µi

return µ1, µ2 … µc

end

Image Processing Techniques for Image Processing Techniques for pre-processing & evaluationpre-processing & evaluation

• Non-linear Transformations.

• Gray level connectivity.

Non-Linear TransformationNon-Linear Transformation

xI =

2 3 2 4 3 5 6 6 5 4

4xI

=

16 81 16 256 81 625 1296 1296 625 625

Gray Level Connectiveness Gray Level Connectiveness

Both 50% of dark pixels and 50% bright pixels

Gray Level ConnectivenessGray Level Connectiveness

• Classify the lowest set of pixels .1 gray values away from each other as radiodense.

• Afterwards, all regions were analyzed for connectiveness by classifying regions as connected as long as they were within .1 gray values of each other.

VarianceVariance

Typical Harvard

Typical FCCC

VarianceVariance

Variance of Radiodense Regions

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

1159

9502

1448

0101

1913

1709

2625

3102

2865

7701

0691

7201

0691

7201

2227

2101

2276

5901

0308

3801

Var

ian

ce

VarianceVariance

Variance of Radiolucent Regions

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

1159

9502

1448

0101

1913

1709

2625

3102

2865

7701

0691

7201

0691

7201

2227

2101

2276

5901

0308

3801

Var

ian

ce

Gabor FiltersGabor Filters

Typical Harvard

Typical FCCC

Co-occurrence MatrixCo-occurrence Matrix

Results of clustering

Expected result

Law’s EnergyLaw’s Energy

Non-linear TransformationNon-linear Transformation

ConnectivenessConnectiveness

Texture Conclusions Texture Conclusions

• Characteristics of radiodense and radiolucent tissue vary from image to image as well as region to region.

• The two regions have similar characteristics. So similar that separability seems unlikely.

Local Contrast Estimation (LCE)Local Contrast Estimation (LCE)

• Image Preprocessing• Tissue Segmentation

Mask• Compensation for

Compression• Threshold Selection

based on Local Contrast Estimation.

Tissue Segmentation

Compression Mask

Threshold Based on Global Estimate of Range

Image Compression Adjustment

Image Pre-Processing

Radiodensity Estimation of Mammogram

Image PreprocessingImage Preprocessing

Stripes caused during mammogram scan

Image ProcessingImage Processing

50 Percent of Left Side

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Mean.3715

1.5 SigmaThreshold

Statistical analysis

Binary Segmentation

Stripe Removal

Tissue SegmentationTissue Segmentation

• SV-CNP – Bimodal Histogram Analysis through block processing and RBF generalization.

• LCE- Image Morphology and RBF generalization.

Tissue SegmentationTissue Segmentation

(a) (d)(c)(b)

Previous method of tissue segmentation

Tissue SegmentationTissue SegmentationPrevious method of tissue segmentation


Harvard FCCC


The white stripes were sometimes brighter than the tissue itself

Tissue Segmentation Tissue Segmentation (Morphology)(Morphology)

Tissue Segmentation Tissue Segmentation (Generalization)(Generalization)

• Obtain enhanced image.• Group averaging for generalization.• Obtain initial semicircle RBF mask.• Starting from center, check pixel by pixel for

mean square error.• Only include pixels that have Euclidean distance

MSE below threshold for next iteration of mask.• Implement until no changes are made.

Tissue Segmentation Tissue Segmentation (Generalization)(Generalization)

Compensation for CompressionCompensation for Compression

• SV-CNP – Homotopy Continuation algorithm.– Multiple filtering operations.

• LCE– Gaussian interpolation.

Compensation for Tissue Compensation for Tissue Compression (SV-CNP)Compression (SV-CNP)

From mask, obtain boundaries

From, boundaries, obtain family of curves

Multiple filter operations to obtain final image


Scale problem

Resolution problem

Resolution problem


Region A

Region B

Compensation for Tissue Compensation for Tissue Compression (LCE)Compression (LCE)

Line by line interpolation of a Gaussian function

Local Contrast Estimation (LCE)Local Contrast Estimation (LCE)

• Perceive connected regions a layers.

Boundary EstimationBoundary Estimation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

500

1000

1500

2000

2500

grey level values

occu

renc

e

From the mask, the locations of these

artificial boundaries created by threshold

t is then found

Local Contrast EstimationLocal Contrast Estimation

Using threshold t, the mask of the

radiodense regions is created

Step 1: Estimation of the Boundaries

Local Contrast EstimationLocal Contrast EstimationStep 1: Boundary Estimation

Any white pixel in the new boundary mask will correspond a region where the estimated threshold t believes there is a change from radiolucent to radiodense tissue.

Local Contrast EstimationLocal Contrast EstimationStep 2: Calculation of Local Contrast

From this region selected by the boundary mask, a collection of pixels is collected by four methods:

Horizontal

Vertical

Southwest-Northeast Diagonal


Split into two groups,

7 numbers higher than median

7 number lower than median

The highest function of 4 methods

= local Edge Function

Local Contrast EstimationLocal Contrast EstimationStep 2: Calculation of Local Contrast

For each collection method:

Horizontal Vertical Southwest-Northeast Diagonal


: a local contrast estimation is calculated

Calculate median

Find Difference of two group means

MH - ML

Local Contrast EstimationLocal Contrast EstimationStep 3: Calculation of Global Contrast

After the local contrast estimation is obtained for all regions defined by the mask, and average global estimate is obtained.

N

iContrastContrast

N

iLocal

Global

1

)(

)()()(

))(max(

LowH

Local

GroupmeanGroupmeaniContrast

iContrastContrast

where i is one of four collection methods

N being the total number of Local Contrasts

• A sweep of thresholds is done for each image.

0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6 0.62 0.64 0.660.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

Local Contrast EstimationLocal Contrast EstimationStep 4: Calculation of Optimum Contrast

Glo

bal

Con

tras

t

Threshold

(Only small region is being shown in graph)

Based on graph, the threshold with the highest global contrast is chosen as the optimum threshold

ResultsResults

• Databases.

• Scanners.

• LCE

• LCE vs. CNP vs. SV-CNP.

DataDataBrigham and Women’s HospitalHarvard Medical SchoolCambridge, MA

Brigham and Women’s HospitalHarvard Medical SchoolCambridge, MA

Fox Chase Cancer CenterPhiladelphia, PAFox Chase Cancer CenterPhiladelphia, PA

Rowan UniversityGlassboro, NJRowan UniversityGlassboro, NJ

Digitized bmpImages (10)

Mammogram X-Ray Films

Digitized jpg & dicomImages (717)

FRAP (339) Chinese-American (378)

Digitizers:Agfa (OD = 3.0)Lumisys (OD = 3.85)


Scanner ComparisonsScanner Comparisons

AGFA Scanner Lumisys Scanner

Local Contrast EstimationLocal Contrast Estimation

• Image Preprocessing• Tissue Segmentation

Mask• Compensation for

Compression• Threshold Selection

based on Local Contrast Estimation.

Tissue Segmentation

Compression Mask

Threshold Based on Global Estimate of Range

Image Compression Adjustment

Image Pre-Processing

Radiodensity Estimation of Mammogram

Image PreprocessingImage Preprocessing

Tissue Mask SegmentationTissue Mask Segmentation

Compression Compensation MaskCompression Compensation Mask

Comparison between 3 methodsComparison between 3 methods

Problems with CNPProblems with CNP

2212

221

CNPT

Supervised Parameter

Problems with SV-CNPProblems with SV-CNP

• Based on threshold from CNP….

• Compression values over fit data to correlate with percentages.

• Final segmentation results do not visually match with the expected segmentation.

Problems with SV-CNPProblems with SV-CNP

Harvard ResultsHarvard Results

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

11051702 11599502 14480101 15839502 19131709 20110811 26253102 26799401 2778620 28657701

TorontoCNPSV-CNPLCE

Harvard ResultsHarvard Results

91% 1091

92% 495

87% 879

CNP

SV-CNP

LCE

MSECorrelation











34 selected validation images

CNP FCCCCNP FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0Toronto

CNP

SV-CNP FCCCSV-CNP FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0Toronto

SV-CNP

LCE FCCCLCE FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

Toronto LCE

FCCC ResultsFCCC Results

-39% 21732

48% 15127

84% 4052

CNP

SV-CNP

LCE

MSECorrelation











34 selected validation images

Combined ResultsCombined Results CNP SV-CNP LCE

Correlation compared to Toronto method (with

flagged) 0.147 0.565 0.855

Correlation compared to Toronto method (without flagged) 0.306 0.733 0.882

MSE compared to Toronto method (with

flagged) 22823.930 15622.682 4931.374

MSE compared to Toronto method (without flagged) 14381.540 5425.792 2811.690

Average % difference compared to Toronto method (with flagged) 18.186 12.924 8.232

Average % difference compared to Toronto

method (without flagged) 17.889 8.791 7.927

Database ResultsDatabase Results

182 27.5%

133 20%

87 13%

73 11%

50 7.5%

32 4.8%

10 1.5%

0 0%

0 0%

0 0%

Out of 660 images

0%-10%

10%-20%

20%-30%

30%-40%

40%-50%

50%-60%

60%-70%

70%-80%

80%-90%

90%-100%

# images % of database

14% of the images could not be evaluated

Database ResultsDatabase Results

37.9% 23.9%

19.5% 19.5%

9.8% 14%

10% 11%

5.7% 9.9%

1.4% 3.3%

.3% 1.5%

0% 0%

0% 0%

0% 0%

0%-10%

10%-20%

20%-30%

30%-40%

40%-50%

50%-60%

60%-70%

70%-80%

80%-90%

90%-100%

FCCC Chinese American

Database IssuesDatabase Issues

• 93 images could not be analyzed.

Summary of AccomplishmentsSummary of Accomplishments

• Development of a comprehensive database from multiple age and ethnic groups.

• Development of a completely automated radiodense tissue segmentation procedure.

• Comparison of new method with a previously established segmentation method.

• Algorithm has the ability to sift through entire databases of digitized mammograms quickly.

ConclusionsConclusions

• LCE is able to give good performances across multiple databases without the need to supervise.

• LCE is fully automated.• LCE is 86% correlated with an established

method• The average difference in percentage is less

than 8.3%

IssuesIssues

• Tissue segmentation algorithm still has trouble generating accurate boundaries for low contrast images.

• For images that had multiple layers of gray level intensities, the algorithm has no clue which layer is supposed to be chosen to achieve the estimate comparable to the radiologist’s.

Recommendations for Future Recommendations for Future WorkWork

• Canonical images for radiodensity segmentation.

• Accurate model for tissue compression.

• Mammograms from the scanned using the Agfa should be scanned again.

AcknowledgementsAcknowledgements

• Dr. Marilyn Tseng, FCCC

• Dr. Celia Byrne

• Dan Barrot

• Lyndsay Burd

• Any questions before I get married………???

thesis advisor: dr. mandayam committee: dr. kadlowec and dr. polikar

Documents

cases of cancer

respiratory system7030025

respiratory system8360012

diges system12010018

digestive system6170022

br myeloma68001

soft tissue19000

s tissue38000