hourglass shapes in rank grey-level hit-or-miss …...hourglass shapes in rank grey-level...

Hourglass shapes in rank grey-level

hit-or-miss transform for membrane

segmentation in HER2/neu images

Marek Wdowiak, Tomasz Markiewicz, Stanislaw Osowski, Zaneta Swiderska,

Janusz Patera, Wojciech Kozlowski

Warsaw University of Technology, Military Institute of Medicine, Military University of Technology, Warsaw, Poland

• Segmentation of the thin, non-continuous and highly

variable objects is a difficult task in mathematical

morphology application.

• To such problems belongs the membrane segmentation of

the cells in histopathology Human Epidermal Growth

Factor Receptor 2 (HER2/neu) images related to the breast

cancer.

• The histopathological evaluation of such

immunochemistry stains specimens is the most common

task for pathologists.

Introduction

| Introduction | Methods | Results | Conclusion | Acknowledgement |

Introduction

Four categories in grade scale are recognized:

• 0 (no membrane staining is observed or membrane staining is

observed in less than 10% of the tumor cells),

• 1+ (a barely perceptible membrane staining is detected in more

than 10% of tumor cells, the cells exhibit incomplete membrane

staining),

• 2+ (a weak to moderate complete membrane staining is observed

in more than 10% of tumor cells),

• 3+ (a strong complete membrane staining is observed in more

than 10% of tumor cells).


Introduction

As shown, the categorization

criteria are very subjective and

may lead to the significant

differences in assessment of

the particular cases. Especially

the distinction between weak

and strong, and the continuity

of a membrane staining may

result in a large variation in

their interpretation.


Methods – the proposed algorithm of image processing

The image processing pathway was composed on:

a) enhancement of an image contrast,

b) creating a set of different colour space representations and

classification of image pixels into the nuclei, membrane

with positive reaction, and the remaining regions,

c) segmentation of the cell nuclei,

d) recognition of the stained cell membranes based on the

modified rank grey level hit-or-miss transform with the

new proposed shape patterns,

e) allocation of the parts of membranes to the separate cells.


| Scheme of the Algorithm | Enhancement | Learning Data | Cell Segmentation | Membrane Segmentation | Hourglass Shapes | Hit-or-Miss |

Methods – enhancement and colour representations

• The contrast enhance was realized by applying an automatically

computed contrast stretch and normalizing the colour components

to fulfill their ranges.

• Next, the nine sub-image samples were prepared and manually

segmented into the blue nuclei (class 1), reactive brown

membrane (class 2), and the remaining areas (class 3) to establish

the most adequate pixel colour components for a classifier.

• The following colour spaces were taken into account: RGB,

CMYK, HSV, YCbCr, CIE Lab, CIE Lch, CIE uvL, CIE XYZ.



Methods – enhancement and colour representations

• Their abilities to differentiate the specific classes were evaluated

using an area (AUC) under Receiver Operating Characteristics

(ROC) curve .

• Also, the cross-correlation between the candidate features was

studied in order to select not only the best ones, but also with the

least correlation between themselves.

• Finally, a set of six features composed of two the best ones for

each class was found and used as a feature vector for a classifier.

• The classification was performed using the SVM with a Gaussian

kernel function.



Methods – learning data

The original images and

manually marked classes

comes from the nine

samples of HER2/neu

image as the learning

data.



• The binary nuclei mask obtained from the

classifier is used for the cell nuclei segmentation.

• The morphological closing operation was

applied to reduce a noise.

• For separation of the connected nuclei the

distance map was build and each extended

regional maxima with the selected h value was

recognized as the centre of cell nucleus.

• Finally, the restriction to a single cell nucleus

area was applied to eliminate the non-cancer cell

masks.

Methods - segmentation of cell nuclei



Methods - intensity map for membrane segmentation

• The crucial input signal to this step

can be an intensity map restricted

comes from a single colour channel,

e.g. luminance or yellow

component, as well as composed

from a set of colour channels.

• The presented sample intensity map

is the element-wise product of Y

channel from CMYK and u channel

form CIE uvL colour space.



Methods – novel hourglass shapes for discovering the line structures

• To discover the line structures the appropriate

composed structuring elements should be

applied.

• Due to a high variability of a cell membrane

geometry, a flexible shapes are necessary.

• We proposed and defined the novel hourglass

shapes as they are presented on the figures.

• The central region (marked on red) is treated as a

foreground (FG), the hourglass shape is marked

in grey (HG), and the white areas represent

background (BG).



Methods – novel hourglass shapes for discovering the line structures

• The second set of shapes were the

half-hourglass structures presented

on the right.

• When the two hourglass shapes

oriented horizontally and vertically

covered the most cell membrane

orientations, the half-hourglass are

in four directions and each of them

required a highest correspondence

of membrane segment shape with

them.



Methods – grey-level hit-or-miss transforms – related works

In the literature the most closely to the examined problem are:

• Blur grey-level hit-or-miss transform (Bouraoui et al., 2010) ,

defined as:

𝜂[𝑉,𝑊;𝐺,𝐻]𝑠 𝐼 𝑝 =

𝐼 ⊕ 𝐺 ⊖ 𝐴 𝑝 − 𝑎 𝑖𝑓 𝐼 ⊕ 𝐺 ⊖ 𝐴 𝑝 − 𝑎

≥ 𝐼 ⊖ 𝐻∨ ⊕ 𝐵∨ 𝑝 − 𝑏

⊥ 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

where the dilation eliminates dark noise in the bright FG, while the

erosion 𝐼 ⊖ 𝐻∨ eliminates bright noise in the dark BG.



Methods – grey-level hit-or-miss transforms – related works

The other extension of HMT is the rank (gray-level) hit-or-miss

(Soille, 2003):

[𝑈𝐻𝑀𝑇 𝑓 ] 𝑥 =

𝜁𝐵𝐹𝐺𝑘𝐹𝐺 𝑓 𝑥 − 𝜁𝐵𝐵𝐺𝑘𝐹𝐺′ 𝑓 𝑥 𝑖𝑓 𝜁𝐵𝐵𝐺𝑘𝐹𝐺

′ 𝑓 𝑥

< 𝜁𝐵𝐹𝐺𝑘𝐹𝐺 𝑓 𝑥

0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

where the rank vector determines the uncompleted inclusion of

FG/BG in the spatial location on the image.

This formulation is the most closely to requirement of our problem.



Methods – grey-level hit-or-miss transforms – our modification

In our study we propose a slight modification to the Soilles’ UHMT,

that can be defined in the form:

where the result is in a binary form directly defined on the grey-

level by the h value. Instead of a rank filter, for the FG region we

applied the average value as a reference level.

Finally, a watershed algorithm is performed to established the cell

contours and allocation of membranes in the separate cells.

hXXHMT FGkBhk BGBGBG ,,,B



Methods – the proposed scheme of the algorithm



Results – material

• The materials used in experiments come from the archive of the

Pathomorphology Department in the Military Institute of Medicine in

Warsaw, Poland.

• Twenty cases of the breast cancer of HER2/neu preparations without

any artefacts representing 1+, 2+, and 3+ grades were selected. The

paraffin embedded breast tissue were stained in a standard way

according to the Ventana PATHWAY anti-HER-2/neu (4B5) Rabbit

Monoclonal Primary Antibody protocol.

• The specimen images were registered on the Olympus BX-61

microscope with the DP-72 colour camera under the magnification

400x and resolution 1024x768 pixels.


| Material | Feature Selection | Hourglass Shape Application | Cell Segmentation | Graphical Representation | Numeric Results |

Results – feature selection for the classification process

On the basis of the

presented results the

following features have

been selected: R and B

components from RGB

space, u and L

components from uvL

representation, b

component from CIE

Lab and Y component

from CMYK.



Results – hourglass shapes for HMT

The hourglass shapes

were adjusted to: FG

size 3×3 pixels, the

distance between FG

and BG equal 4

pixels, mask size

21×21 pixels.

The rank value was

equivalent of 80 %,

the normalized h

value was 0.2



Results – half-hourglass shapes for HMT

The half-hourglass

shapes were

adjusted to: FG size

3×3 pixels, the

distance between

FG and BG equal 4

pixels, mask size

21×21 pixels.

The rank value was

equivalent of 80 %,

the normalized h

value was 0.2 | Introduction | Methods | Results | Conclusion | Acknowledgement |


Results – cell segmentation and allocation of membranes in separate cells

• Next, based on the

watershed, the cell

outlines were found.

• Finally, the recognized

immunoreactive

membrane segments

were allocated to the

specific cells.



Graphical Results

Expert Algorithm Expert Algorithm Expert Algorithm



Graphical Results



Numeric Results

The numerical

results of

membrane

continuity

estimation made by

expert and our

system at

application of

hourglass and half-

hourglass shapes.

Case Continuity

Expert’s result Hourglass Error(%) Half-hourglass Error(%) 1 0,404 0,383 -0,020 0,351 -0,053

2 0,316 0,367 0,051 0,335 0,019

3 0,293 0,423 0,131 0,388 0,096

4 0,359 0,534 0,175 0,474 0,116

5 0,316 0,371 0,055 0,332 0,016

6 0,304 0,446 0,142 0,401 0,097

7 0,322 0,330 0,008 0,289 -0,033

8 0,353 0,378 0,025 0,339 -0,014

9 0,232 0,309 0,077 0,279 0,047

10 0,328 0,450 0,122 0,409 0,080

11 0,308 0,380 0,072 0,343 0,036

12 0,191 0,295 0,104 0,271 0,080

13 0,256 0,339 0,083 0,314 0,058

14 0,298 0,431 0,134 0,396 0,098

15 0,373 0,490 0,116 0,447 0,074

Mean absolute error 8.8% 6.1%

Standard deviation 4.8% 3.2%



Conclusion

• The new approach to the automatic evaluation of the HER2/neu

membrane staining in the breast samples was proposed.

• The important point in this approach is the application of the

hourglass-shape structuring elements in the rank grey-level hit-

or-miss transform for the analysis of the image.

• The experimental results have shown high efficiency of image

segmentation with respect to the nuclei and membrane

localizations.


Acknowledgement

This work is supported by the National Centre for Research and Development (Poland) by the grant PBS2/A9/21/2013 in the years 2013-2016. Thank you very much for your attention.


hourglass shapes in rank grey-level hit-or-miss …...hourglass shapes in rank grey-level...

Documents