A local fuzzy thresholding methodology for multiregion image segmentation

Santiago Aja-Fernandez

LPI, ETSI Telecomunicacion, Universidad de Valladolid, Spain

Ariel Hernan Curiale

LPI, ETSI Telecomunicacion, Universidad de Valladolid, Spain

Gonzalo Vegas-Sanchez-Ferrero

LPI, ETSI Telecomunicacion, Universidad de Valladolid, Spain

Abstract

Thresholding is a direct and simple approach to extract different regions from an image. In its basic for-mulation, thresholding searches for a global value that maximizes the separation between output classes.The use of a single hard threshold value is precisely the source of important segmentation errors in manyscenarios like noisy images or uneven illumination. If no connectivity or closed objects are considered, themethod is prone to produce isolated pixels. In this paper a new multiregion thresholding methodology ispresented to overcome the common drawbacks of thresholding methods when images are corrupted withartifacts and noise. It is based on relating each pixel in the image to different output centroids via a fuzzymembership function, avoiding any initial hard decision. The starting point of the technique is the defini-tion of the output centroids using a clustering method compatible with most thresholding techniques in theliterature. The method makes use of the spatial information through a local aggregation step where themembership degree of each pixel is modified by local information that takes into account the membershipsof the surrounding pixels. This makes the method robust to noise and artifacts. The general formulation ofthe proposed methodology allows the design of spatial aggregations for multiple applications, including thepossibility of including heuristic information via a fuzzy inference rule base.

Key words: thresholding, fuzzy, segmentation

1. Introduction

Thresholding is one of the most direct and simple approaches to image segmentation. It is an effectivemethod as long as the image shows well defined areas and the gray levels are clustered around distant valueswith minimum overlap. It also has been used to provide an initial estimation or a prior to more complexsegmentation methods (techniques based on snakes, level-sets or active contours need an initial segmentation,that can be manually done or obtained via thresholding [1, 2]), to provide masks of regions of interest [3],or even as a technique to detect motion in surveillance environments [4, 5]. Thresholding is also extensivelyused in the medical imaging field, where images are composed by several tissues, represented by their graylevels [6]. The arrangement of these tissues or organs inside the image is usually clearer than the arragementof objects in a natural scene image, hence the using of specific thresholding techniques.

Image thresholding techniques are well known, and some of the most used methods date from the 70s,such as Otsu’s method [7, 8]. In its basic implementation, thresholding methods search for a global threshold

Email addresses: sanaja@tel.uva.es (Santiago Aja-Fernandez), ariel@lpi.tel.uva.es (Ariel Hernan Curiale),gvegsan@lpi.tel.uva.es (Gonzalo Vegas-Sanchez-Ferrero)

Preprint submitted to Knowledge-Based Systems January 26, 2015

value that somehow maximizes the separation between classes in the final result. However, regardless of themethod chosen to find the separation between classes, the use of a single hard value is known to be the sourceof important segmentation errors when dealing with noisy images, uneven illumination and soft transitionsbetween gray levels [9, 10, 11]. The main drawback of this global threshold approach is due to it being pixeloriented rather than region oriented, and therefore those pixels having the same gray level value will alwaysbe segmented into the same class. If no connectivity or closed objects are considered, the method is proneto produce isolated pixels.

Thus, despite being a long standing problem, these issues have not been resolved, and new approaches arerequired to solve different configuration of signal and images; see some surveys of them in [9, 12, 13, 14, 15].In the first one [9], authors classify thresholding methods into six main categories:

1. Methods based on the shape of the histogram [7, 8].

2. Clustering-based methods [16, 17, 18, 19, 20, 21].

3. Entropy-based methods [22, 23, 24].

4. Local methods, that adapt the threshold value on each region based on local features [25, 26].

5. Object attribute-based methods that search a measure of similarity between gray-levels and objects,such as fuzzy shape similarity.

6. Spatial methods that use higher-order probability distribution and/or correlation between pixels [11].

The first three methodologies comprise the main tradition behind thresholding methods: the search fora global threshold that allows us to divide the image into two or more regions. Methods proposed in theliterature can grow in complexity in the pursuit of the optimal threshold but ultimately, the final segmentationwill only depend on the gray level of each particular pixel. The final classification is done pixelwise. Notethat most of the algorithms based on fuzzy measures [27, 28, 29, 30, 31, 32, 33] fall into these categories. Onthe other hand, local methods assume that different areas in the image require different thresholds. This isthe case for images with uneven illumination, in which objects are not totally represented by absolute grayvalues.

All these methodologies will fail in the case of images corrupted by noise, where the gray levels of eachobject are spread and merged due to the noisy distortions. Attribute-based methods are a good alternative,as long as we have key information about the objects in the scene. Finally, spatial methods take into accountpossible relations between pixels. The idea behind them is the fact that pixels belonging to the same objectwill have a certain degree of connectivity, i.e., the presence of isolated pixels is unlikely and there is a strongrelation between a pixel and its neighborhood.

Note that these six categories can be merged into three practical methodologies:

1. Methods that calculate a global threshold for the whole image.

2. Methods that use an adaptive local threshold.

3. Methods that use spatial local information for classifying the pixels.

In this paper we propose a new thresholding methodology that takes advantage of the main features ofthe last two categories:

• The membership degree of a particular pixel in a class is spatially related with the membership of itssurrounding neighbors.

• The final thresholding will take into account the local membership in each of the classes, which implicitlymakes the threshold locally variant.

2

[0.48 class 2; 0.52 class 3]

Class 1

Class 2

Class 3

Figure 1: Example of classification of noisy pixel.

The main contribution of this paper relies on Fuzzy Sets Theory and Fuzzy Logic [34]. Fuzzy logic isknown to be a very flexible tool in classification problems where imprecise knowledge or not-well-definedfeatures have to be used. In addition, fuzzy logic is also a natural selection where information has to beretrieved from linguistic statements. It has been widely used in the field of systems control [35], but thereare also a great amount of applications in the image processing field [36, 37, 38]. In the last 20 years, manymethods based on fuzzy logic and fuzzy measures [39, 40] have been proposed for image thresholding. Theyare mainly focused on the search for the optimum threshold using fuzzy measures, buy many times they donot take into account spatial information. Some of the techniques used comprised fuzzy clustering [41, 42],modified versions of fuzzy clustering methods [19, 21], fuzzy measures [27, 43, 30, 44], optimization of fuzzycompactness [45], fuzzy entropy [46] and the interpretation of thresholds as type II fuzzy sets [33, 32]. Parallelwith fuzzy measures, other soft computing methods have arisen, such as the heuristic methods based on ant,bees and bacteria colonies [47, 48, 49].

In this paper we propose a new methodology which differs from those approaches in the literature. Thestarting point is the idea that the membership of a pixel in a particular class or object will be highlycorrelated with the membership in that class of the surrounding pixels. To take into account this localspatial information we propose the use of fuzzy sets: a pixel will be assigned to the different classes of amultiregion segmentation through a fuzzy membership function. The traditional hard assignment (i.e. apixel belongs or does not belong to an output class) is replaced by a soft assignment, following the basictheory of fuzzy sets.

The paper is organized as follows: Section 2 presents the new thresholding methodology proposed. Resultsand comparison with another methodologies and methods are shown in Section 3. Conclusions are presentedin Section 4.

2. Multiregion Fuzzy Thresholding

2.1. Motivation and purpose

The main limitation of global thresholds is that pixels having the same intensity levels will always besegmented into the same class. This may lead to misclassification in images corrupted by noise or unevenillumination. To overcome this problem, information about the behavior of the spatial surroundings of eachpixel must be considered. This spatial information can be taken into account using very different methods,each one of them generating a different output segmentation. Most used methods are those we can call blindmethods: methods that clean the segmented image using local processing but without any prior information ofthe image structure, object distribution, nature of noise, etc. These methods only make use of the segmentedvalues. Some common examples are the median filtering and morphological operations to remove isolatedpixels.

To motivate use of the local information, see for instance the image in Fig. 1. A pixel has been classifiedas belonging to the Class 3 (red). In a 3× 3 neighborhood around the pixel we can check that this pixel isan isolated value, probably generated by noise. The correction to this misclassification can be done using

3

0 0.2 0.4 0.6 0.8 10

0.2

0.5

0.4

1

0.6

1.5

0.8

2

1

2.5

0

3

0.2

3.5

0.4

4

0.6

0.8

1

I(x)pre−processing

[Membership degrees]

[ParametricValues]

Image

Membership

Local

Aggregation

Original Image

Extraction

Centroid Assign

Segmentation

Image

Thresholded imageDefinition of membership functions

Fuzzy MFs

definition

Local

Rules

Figure 2: Pipeline of the multiregion fuzzy thresholding methodology described in the paper.

information about the image (such as the model of noise, the classification probability, the distance to thecentroid) or just using spatial or morphological operations. Note that if we use a median filter over thatneighborhood, the pixel will now be classified as Class 1 (blue). Similar results can be found using a fillingalgorithm. In those cases, important information about the image is missing. If we check the distance to thecentroids, we realize that the pixel is 0.52% Class 3 and 0.48% Class 2, and it has been classified as 3 by asmall range. In a noisy image, it is very likely that this pixel belongs to Class 2, and is unlikely to belongto Class 1. In what follows we will be using this idea of taking into account the local properties to improvethe classification methodology.

We propose a new thresholding methodology to make a multiregion segmentation of the different areaswithin an image. To that end, we will follow a fuzzy assignment classification that will follow the philosophybehind many fuzzy-based approaches in the literature [27, 28, 29, 30, 31], but it will be complemented witha spatial aggregation step that will take advantage of the soft classification and the spatial relations. Ourfuzzy thresholding methodology will assign a membership degree to every pixel for each of the output classes,rather than to a traditional hard thresholding. The membership degree of each pixel is then modified usinglocal information following some aggregation scheme and some fuzzy rules set beforehand. The aggregationmethod is to be specifically designed for each particular application, although some examples will be given.The inclusion of this aggregation step will be a great advantage when dealing with noisy images.

Finally, note that the methodology is compatible and complementary to some of the methods alreadyproposed in the literature, since the starting point is precisely a fuzzy assignment of pixels to classes.

2.2. Proposed thresholding methodology

Let I(r) be an image with L different regions we want to extract via thresholding, in order to obtain asegmented image M(r), so that

M(r) = gs{I(r)}

where gs{.} is the segmentation method that can be considered a function that maps the NI levels of grayof image I(r) into L values, i.e. gs : NI → L with L < NI .

To carry out the segmentation, the proposed methodology assumes that every pixel in the image I(r)will have a degree of membership in each of the L regions. That membership will be modeled using fuzzymembership functions. In what follows, we will denote µl(x), l = 1, · · · , L, as the fuzzy membership functionof the lth area. Spatial aggregations can therefore be applied into the membership plane to take into accountthe relation between adjacent pixels prior to obtain the final segmentation.

The methodology, surveyed in Fig. 2, comprises five steps. Since the proposal is a methodology and not aclosed algorithm, in each of the steps we provide alternative implementations of the methods. The electionof any choice will depend on the kind of data and the particular features of the segmented image. Someof the proposed choices were tested and will be described in the experiments section. The five steps of thethresholding methodology are as follows:

1) Extraction of the centroids: The different areas in which the image will be segmented are definedusing L centroids. Typically, as previously stated, L < NI . The number of centroids can be manually set

4

0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

3.5

4

HistogramFitted Gaussian

(a) Histogram with fitted Sum of Gaus-sian

0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

(b) Gaussian membership functions

0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

(c) PTS membership functions

Figure 3: From probability to membership: (a) (Normalized) histogram of the image h(I), with three weighted Gaussian, pl(x)fitted using MMSE. (b) From pl(x, θ), three fuzzy sets with Gaussian membership functions are created. (c) Alternatively, frompl(x, θ) three fuzzy sets with PTS membership functions are created.

beforehand by the user or the algorithm can be automatically tuned to search for the optimum number ofregions. Many of the methods proposed in the literature for traditional thresholding may be used here inthis first step, specially those based on clustering [16, 17, 50] or entropy of the histogram [23, 24].

2) Definition of fuzzy membership functions: A fuzzy membership function µl(x), with l = 1, · · · , Lis associated to each of the classes linked to the previously defined centroids. There are two possible waysto define the membership functions:

(a) From the histogram of the image, h(I). This method follows the classical approach for multiregionthresholding, in which the main levels within the image are extracted from the histogram, h(I). A sumof L weighted distributions is fitted to h(I):

h(I) ≈L∑l=1

ωl · pl(x;θl) (1)

with pl(x;θl) a probability density function defined by the set of parameters θl, and ωl are weights whichsatisfy that

∑l

ωl = 1. The fitting can be done using a minimization algorithm, such as minimum mean

square error (MMSE):

arg minθl,ωl

∣∣∣∣∣h(I)−L∑l=1

ωl · pl(x;θl)

∣∣∣∣∣2

. (2)

Typically Gaussian distributions are good candidates for pl(x;θl) to represent the histograms. However,some modalities of medical imaging may benefit by using alternative distributions. MR data, for instanceis known to follow a Rician distribution [51, 52] that can be accurately approximated by a Gaussian forhigh Signal-to-Noise Ratios. Ultrasound data, on the other hand, have been described using a myriadof distributions, such as Rayleigh, K or homodyned-K. Lately in [53] authors show that, due to theinterpolation on the data, the histogram may be represented more accurately by a mixture of Gammadistributions. Therefore, in those applications, a Gamma is a better candidate for pl(x;θl).

Our intention is to use membership instead of probability values. To that end, we use the informationof the histogram to model the fuzzy sets that will provide this membership information. The simplestway would be using Gaussian membership functions (MF), such as the ones in Fig. 3-(b). Note that thetransition from probabilities to membership comes with a transformation of the first and last sets, anda normalization of the weights.

5

Although a natural transition, Gaussian MFs have some problems related to Fuzzy Sets Theory [54, 55]:they are not consistent and the intersection of the sets is in a value smaller than 0.5. In order to workproperly, we require the fuzzy sets to be:

(i) A complete partition: ∀x ∃µl(x), 1 ≤ l ≤ L, such that µl(x) > 0.

(ii) Consistent: if µl(x0) = 1, then µk(x0) = 0 ∀k 6= l.

(iii) Normal: max(µl(x)) = 1.

(iv) The intersection between adjacent fuzzy sets is µl(x0) = µl+1(x0) = 0.5.

To that end, we choose to use Pseudo Trapezoid-Shaped (PTS)[54] membership functions, as depicted inFig. 3-(c). Alternative methodologies to obtain the MFs from the histogram can be found in [56, 57, 58].

(b) From the centroids, leaving aside histogram information. If PTS-MFs are used, all the information ofthe histogram can be left aside, since the sets in Fig. 3-(c) can be built directly from the centroids valuesextracted in the previous step. This is an advisable solution for general purpose segmentation or whenthere is no precise knowledge about the distribution of objects or tissues in the image.

In what follows, the central fuzzy sets are selected to be triangular (a particular shape of PTS). However,more complex shapes may be adopted, as trapezoid MFs, taking into account the variance of the fittedGaussian.

The output of this step will be the MFs that describe each of the output sets, µl(x), 1 = 1, · · · , L.

3) Assigning membership to each pixel: The membership of pixel r in the image I(r) in the l-th classis given by µl(I(r)). Using PTS MF as previously defined, note that

L∑l=1

µl(I(r)) = 1.

At this point, a first thresholding of the image could already be done:

M(r) = arg maxl{µl(I(r))} (3)

with M(r) the output thresholded image. However, this way we are not taking advantage of the informationof the neighborhood, and the results will be strongly dependent on the method selected for the centroidsearch.

The output at this stage, for each pixel, will be a vector of memberships:

µ(I(r)) = [µ1(I(r)) µ2(I(r)) · · · µL(I(r))]

If PTS MFs are selected, only 2 of the elements of each vector would be different from zero.

4) Local information aggregation. In this step, spatial information will be taken into account beforeobtaining the final segmentation. This step is the main contribution of the proposed methodology. Theversatility of fuzzy logic allows us to design many different ways to consider the influence of the neighborhoodfrom the µ(I(r)) degrees previously defined. In what follows we propose an aggregation based solution,suitable for general image thresholding.

If we consider a neighborhood η(r) centered around a pixel r, we can use the membership values µ(I(r))of all the pixels in η(r) to better classify the image into regions. To that end we propose a local aggregationof values:

µS(I(r)) = aggs∈η(r)

{µ(I(s))} (4)

6

where agg{.} is an fuzzy aggregation in a neighborhood η(r). The purpose of this function is the modificationof the original membership functions µl(I(r)) when local information is taken into account. One importantfeature of this operation is that we work in the membership space, not over the image or over the finalsegmentation. Different aggregations based on the pixels in the neighborhood can be defined, properlytuned to the desired output. Some neighborhood based rule-sets for fuzzy image processing have beenpreviously proposed in the literature, see for instance [59, 60, 38, 61], that could be easily adapted to theproposed scheme. Fuzzy morphological operators, like those described in [37, 57] can also be applied. In thefollowing we propose some aggregations for general purpose image thresholding.

(a) Median Aggregation (MedAg): The memberships of each of the pixels in η(r) are aggregated using themedian operator

µSl (I(r)) = median

s∈η(r){µl(I(s))} (5)

Note that the neighborhood η(r) can be oriented in order to better find structures in one particulardirection.

(b) Average aggregation (AvAg): We define an averaging of each MF as

µSl (I(r)) =

∑ri∈η(r)

ωi · µl(I(ri)) (6)

with ωi the weights of the pixels considered. One possible definition would be a square uniform neigh-borhood so that ωi = 1/|η(r)|, with |η(r)| the size of the considered neighborhood. Note that whenusing PTS MFs, only two MFs µl(I) will have values different from zero. Thus, the averaging will highlyreduce the value of those isolated pixels, while keeping the values of the homogeneous areas.

(c) Iterative averaging aggregation (IterAg): In order to better find structures, we can define an iterativeprocedure that averages the membership space. Since a tradeoff between structure finding and smoothingmust be considered, a small averaging window must be defined:

h =1

3

0 0.5 00.5 1 0.50 0.5 0

.The aggregation is then defined as

[µl(I(r))]t+1 = [µl(I(r))]t ∗ h, (7)

with ∗ being spatial convolution and t = 0, ..., T the number of iterations. Finally

µSl (I(r)) = [µl(I(r))]T .

(d) Absolute Maximum (AbMax): The membership of each of the pixels in η(r) is aggregated using themaximum operator:

µSl (I(r)) = max

s∈η(r)(µl(I(s))). (8)

We have avoided any kind on implementation based on rules and fuzzy decision, as some heuristic isalways involved. However, for a particular implementation (for instance to threshold bones in radiographimages to search for a particular disease, or for text identification) one can think on using alternative spatialfilters with fuzzy rules, such as the Russo’s FIRE operators [37].

5) Image segmentation. The last step is to calculate the final segmented image from the modifiedmembership functions. We propose using the maximum operator:

M(r) = arg maxl

{µSl (I(r))

}(9)

although other defuzzification and centroid calculation methods can be used [34].

7

3. Experiments and Results

We experimentally tested the methodology proposed in this paper. Throughout this section, the imagesused are those shown in Fig. 4.

First, we analyze the influence of the centroid extraction method on the final segmentation. As previouslystated, different methods can be used in order to extract the L centroids that will determine the multipleregions within the image. The determination of these centroids is key to the accuracy of the method.However, the choice of the most suitable method will depend on the final application. Most of the clusteringand thresholding methods in the literature can be used for this task. For the sake of illustration, threedifferent simple approaches are considered:

1. The centroids are selected from the L most relevant maxima of the histogram h(I), and PTS member-ship functions are created from this centroids.

2. The centroids are obtained from the fitting of L Gaussian distributions to the histogram of the imageh(I) following eq. (2). Gaussian MF are created from the results.

3. A clustering method (fuzzy c-means [50]) is used in order to obtain L centroids. PTS membershipfunctions are created from these centroids.

For each method, two different inputs are considered for the centroid calculation: the unprocessed image,I(r) and a smoothed version, using a Gaussian kernel with σ = 1.5. From the MFs a simple segmentation isdone by using the maximum membership in eq. (3). For this experiment, no local aggregation is done andL = 5 regions are considered. Images in Fig. 4-(a) are used. Results are shown in Fig. 5.

From the results we can see the great influence of the centroid selection method over the final segmenta-tion. For this particular case, the Gaussian MFs, Fig. 5-(b,e),show a less robust behavior and they greatlybenefit from the smoothing of the data. Clustering, Fig. 5-(c,f), appears to be a robust method whichgoes well with the PTS memberships, and it is rather invariant to the smoothing in the image. The useof the maxima of the histogram, Fig. 5-(a,d), also shows a good behavior, though it is more sensitive toimage smoothing. In any case, note that all the methods succeed in properly separating the cell from thebackground, and even in locating important structure inside the cell. In our comparisons with well-knownthresholding techniques, we use the fuzzy c-means clustering method.

Second, we analyze the behavior of the different aggregations proposed. To that end, we will use thefollowing data: the cell in Fig. 4-(a) (corrupted by additive Gaussian noise with σ = 10 and σ = 20) and theradiographic image in Fig. 4-(h). All pixel ranges are [0,255]. In all the methods L = 5 sets are considered.Results are given in Fig. 6 and Fig. 7.

From Fig. 6, we can raise some interesting issues about the behavior of the different methods. Note thatall succeed in the task of properly identifying the regions within the image, despite the noise. When thenoise increases, MedAg fails to properly identify the edges, since it keeps some of the noisy pattern, whileAvAg and IterAg show a very accurate segmentation with connected objects. The AbsMax works properlyfor the original image but shows undesired effects in noisy data. These differences are less obvious when anon-noisy image is considered, like the finger in Fig. 7. There, all the methods show a similar behavior withsubtle differences.

Third, we compare the proposed method with some well-known thresholding techniques. To that end,we have selected some classic algorithms, fuzzy-based thresholding techniques and methods that take intoaccount spatial information. We consider these methods to be representative of the state-of-the-art thresh-olding techniques:

1. Otsu hard thresholding method for 2 regions (Otsu) and an extension for N regions (Otsu-N) [7, 8].

2. Fuzzy c-means (FCM): The segmentation is done using the clustering method proposed in [50]. Thealgorithm clusters objects into c partitions, attempting to find the centroids of natural clusters in thedata. To do that, the algorithm minimizes the intra-cluster variance through a squared error function.The number of centroids is required as a parameter of the method.

8

3. The Iterative Thresholding Segmentation (ITS) algorithm proposed in [10]. We choose this methodsince it was originally for segmentation of very noisy cell images. The initial threshold is iterativelyadjusted based on local information, obtaining a final image less sensitive to noise. Only two outputsets are considered.

4. Maximum a posteriori spatial probability segmentation (MASP), as proposed in [11]. The method isalso intended to be used with noisy images. The method performs pixel-by-pixel segmentation takinginto account local spatial information. The probability of a pixel belonging to a particular class underspatial constraints is defined as the a posteriori spatial probability. The maximum a posteriori is thenused to classify each pixel.

5. Spatial fuzzy clustering (S-FCM), proposed in [18]. This method is proposed to overcome the problemsof standard FCM algorithms when noise and artifacts are present. They incorporate local intensityand spatial information and they can even apply morphological operations and spatial restrictions ata postprocessing stage. In [21], the authors added a level-set step to the method that improves thesegmentation. We confine our experiments to the thresholding step, leaving level sets aside1.

6. Image thresholding using type II fuzzy sets (FT-II), as proposed in [33]. The method calculates aglobal threshold by the minimization of a new measure called ultrafuzziness. It considers the thresholdas a type II fuzzy set. Only two output regions are considered.

We will select three of the proposed aggregation methods, MedAg and AvAg and IterAg, with clusteringfor centroid selection. For the sake of comparison, the thresholding without aggregation in eq. (3) is alsoshown. We will denote it by Max. The following images are used for comparison: the phantom in Fig. 4-(d)(corrupted by additive Gaussian noise with σn = 30 and σn = 80), the cell in Fig. 4-(a) (original image, anoisy version, corrupted by additive Gaussian noise with σn = 10 and a version corrupted by multiplicativeRayleigh noise) and the liver CT image in Fig. 4-(g). For the phantom image, L = 3 regions will be consideredand for the cell and liver images L = 5. Results are shown in Fig. 8 and Fig. 9.

Results for the noisy phantom, Fig. 8, show the inability of some of the classical methods to cope withnoise in the image. If the behavior of the neighborhood is not taken into account, the variations of intensitydue to noise may be translated into different output regions. This is the case for Otsu-N, FCM and MASP.The 2-region Otsu method and the ITS show good results in preserving the central area as a homogeneousregion, although ITS, due to the spatial operations, totally eliminates the inner lines. FT-II shows a morenoisy behavior inside the central regions, but the shapes (and in particular the lines) are better defined.A simple median operator may properly fix this result. A better segmentation can be seen in S-FCM forσ = 30, due to the spatial constraints of this method. However, it cannot cope with high noise levels.

The proposed methods are the ones undoubtedly showing the best results. IterAg of the moderate noisecase is able to identify the 3 different regions without any noisy interference. MedAg and AvAg show slightlyworse results, but, nevertheless, much better that the rest. This differences are accentuated when the highnoise case is considered. Most of the methods are not able to cope with the pattern, even those that usespatial information, like S-FCM. In those cases, methods based on fuzzy aggregations are the only ones thatare able to identify the regions. Even when the segmentation is not totally perfect, AvAg and IterAg succeedin identifying the different areas. The Max results show the thresholding when no aggregation method isused. Note that the noisy pattern present in this image is totally removed by the aggregation methods. Itis a clear illustration of the potential of taking into account the neighborhood with a properly tuned fuzzyrule set.

Results for the cell image, Fig. 9, are also illustrative. First, when no noise is added to the image, most ofthe methods perform similarly, except the ITS that shows a great smoothing. The methods based on fuzzyaggregations show more regular edges and no isolated points, but, nevertheless, results of every method are

1The original MATLAB software provided by the authors carries out a Wiener filtering to remove noise form the image. Totest the method without any additional filtering, this step has been removed.

9

valid. However, great differences arise when images are corrupted by additive noise. Non-fuzzy techniquesshow a more noisy pattern and regions are no longer closed and most methods are not able to identify thebackground. On the other hand, fuzzy methods with aggregations are those with the best results, verysimilar to the non-noisy case. MedAg still shows some isolated pixels but, again, the three fuzzy methodsare able to properly find well-defined closed structures inside the image. These results consistently repeatedfor multiplicative noise. In that case, the background is easily identified by most of the methods, but thedifferent areas inside the cell are mostly erroneous. The proposed algorithms are the only ones showing acapability of properly identifying these inner areas.

Finally, this behavior is corroborated by results for the liver. Most methods are able to properly identifythe different tissues, but the fuzzy aggregations, once more, show more connected results.

Last, for the sake of quantitative comparison, the multilevel thresholding methods are now comparedusing the Berkeley Segmentation Dataset and Benchmark (BSDS300) [62]. For this experiment, the trainingset of 200 color images provided by the data base is considered. As ground truth, the BSDS300 providessegmentations of each image carried out by several human subjects. For the sake of simplicity and compati-bility with the methods in the literature, all the images have been converted to gray scale. In addition, sincethe proposed methodology is intended to work in noisy environments, the images have been corrupted withGaussian noise (σ = 30) and with multiplicative Rayleigh noise, and the experiment was repeated.

The following scalar measures were considered for region-based comparison:

• Rand Index (RI): [63] This is a measure of the similarity between two data clusterings. RI comparesthe compatibility of assignments between pairs of elements in two clusters by calculating the fractionof correctly classified (respectively misclassified) elements to all elements. For two clusterings C1 andC2 it is defined as

RI(C1, C2) =2(n11 + n00)

N(N − 1)(10)

where N is the total number of points, n11 stands for the number of pairs that are in the same clusterin C1 and C2, and n00 is the number of pairs that are in different clusters. The RI has a value between 0and 1, with 0 indicating that the two data clusters do not agree on any pair of points and 1 indicatingthat the data clusters are exactly the same.

• Variation of Information (VI): [64] This measures the distance between two segmentations in terms oftheir average conditional entropy given by:

VI(C1, C2) = H(C1) +H(C2)− 2I(C1, C2) (11)

where H(Ci) is the entropy associated with clustering Ci and I is the mutual information between twoclusterings C1 and C2.

• The covering of a segmentation S1 by segmentation S2 as defined in [65]:

C(S2 → S1) =1

N

∑R1∈S1

|R1| maxR2∈S2

O(R1,R2) (12)

where O(R1,R2) is the overlap between regions R1 and R2 defined as:

O(R1,R2) =|R1 ∩R2||R1 ∪R2|

(13)

We will use two complementary descriptors, C(S2 → S1) and C(S1 → S2).

For each method and for each image, we consider different segmentations using 3 to 10 regions. The bestmatch, in terms of best benchmark results, is considered. Results are shown in Table 1. Note that, for RIand the overlap measures, the greater the better, while for VI the smaller the better. The best value foreach row is highlighted.

10

Otsu-N FCM S-FCM MASP MedAg AvAg IterAg

RIOriginal 0.7138 0.7161 0.7179 0.6721 0.7198 0.7237 0.7241Gaussian 0.6816 0.6862 0.6902 0.6425 0.7075 0.7026 0.7029Rayleigh 0.6325 0.6704 0.6598 0.5803 0.6918 0.6538 0.6502

VIOriginal 2.9746 2.9914 2.9221 3.4835 2.9483 2.8912 2.8820Gaussian 3.3867 3.3964 3.2193 3.9647 3.0776 2.9473 2.9216Rayleigh 3.4821 3.5300 3.2588 3.8401 3.2903 2.9911 2.931

C(S2 → S1)Original 0.3535 0.3499 0.3602 0.2943 0.3558 0.3644 0.3658Gaussian 0.2862 0.2845 0.3121 0.2305 0.3331 0.3517 0.3552Rayleigh 0.2557 0.2506 0.2765 0.2251 0.2951 0.3332 0.3400

C(S1 → S2)Original 0.4414 0.4365 0.4503 0.3414 0.4424 0.4493 0.4507Gaussian 0.3638 0.3614 0.3960 0.2662 0.4191 0.4447 0.4480Rayleigh 0.3526 0.3437 0.3935 0.2933 0.3816 0.4438 0.4554

Table 1: Region benchmarks on the BSDS300. Three experiments are considered: (1) Original Data base; (2) Corrupted byGaussian noise; (3) Corrupted with multiplicative Rayleigh noise. For each configuration, the best value is highlighted.

Results show that the proposed methods are those with the best values on the four considered measures.What is more, they also show a great robustness when the noise experiments are carried out. Note, forinstance, the small variability of the results for the overlap measures in IterAg when compared with otherthresholding methods.

4. Conclusions

We have presented a new thresholding methodology. This methodology overcomes some of the commondrawbacks of thresholding methods when images are corrupted with artifacts and noise. It is related topublished spatial-based thresholding methods, but it also presents some important differences from them,being also related to fuzzy-based strategies. The philosophy behind the proposed methodology is simple: innoisy images the intensity value of a pixel should not be an absolute classification feature, since noise willgenerate similar intensity levels in different objects, leading to a misclassification of isolated pixels. Instead,some metric based on the intensity levels must be considered and this metric must be weighted by theinformation of the surrounding pixels. For this task we have proposed the use of fuzzy membership degrees.Thus, each pixel will belong to different output classes with a certain degree. This fuzzy membership couldalso be replaced by distance to centroids, for instance. The key of the method is to work in the membershipspace rather than on the image space. This way, the memberships of the pixels in a local neighborhoodmodify the membership degree of every particular pixel.

The whole process presented here should be seen as a methodology rather than a closed algorithm, andeach of the different steps can be replaced by equivalent operations. The starting point is the definitionof the centroids for the output classes. In the experiments section we chose to use an FCM algorithm forits robustness and good results. The purpose of this step is to achieve well defined output centroids, andother FCM algorithms have also achieved excellent results, as in [19, 20, 18]. Alternatively most of thethresholding and clustering methods in the literature can be used to carry out this task, so we can benefitfrom very accurate optimization methods that have been used for many years.

The key of the proposal is the use of the the spatial aggregation step, which makes our method able toovercome noise-related problems that traditional methods cannot cope with. Our first attempts to definegeneral purpose thresholding aggregations have shown good results, and can potentially get even better whenapplied with constraints suitable for particular segmentation goals.

All in all, the methodology proposed has the following advantages: (1) It is totally automatic, anddoes not require human intervention; (2) the hard thresholding decision is postponed to the final stage,so all the spatial operations are done before taking into account the different memberships; (3) spatial

11

aggregations make the thresholding more robust to noise and artifacts; (4) it can benefit from the use ofaccurate thresholding methods in the first step; and (5) the spatial aggregations can be specifically designedfor a particular application, and heuristic information can be added via a fuzzy inference rule base.

Acknowledgements

The authors want to acknowledge Prof. Tizhoosh Dr. Li and Dr. Xiong for providing their MATLAB codefor the experiments. This work has been partly funded by Ministerio de Ciencia e Innovacion under researchgrant TEC2013-44194 and by the Instituto de Salud Carlos III under Research Grant PI11-01492. A.H. Cu-riale is funded by the MECyT (Argentina), Fundacion Carolina (Spain) and Universidad de Valladolid FPI-Uva. Authors are with the LPI, ETSI Telecomunicacion, Universidad de Valladolid (Spain). The MATLABfiles of the proposed methods can be found at http://www.mathworks.com/matlabcentral/fileexchange/36918-soft-thresholding-for-image-segmentation.

References

[1] S. Taheri, S. Ong, V. Chong, Level-set segmentation of brain tumors using a threshold-based speedfunction, Image and Vision Computing 28 (1) (2010) 26 – 37.

[2] D. Mukherjee, N. Ray, S. Acton, Level set analysis for leukocyte detection and tracking, IEEE Trans.Image Proccess. 13 (4) (2004) 562–572.

[3] J. Sijbers, A. Postnov, Reduction of ring artefacts in high resolution micro-CT reconstructions, Physicsin Medicine and Biology 49 (14) (2004) 247.

[4] P. L. Rosin, Thresholding for change detection, Computer Vision and Image Understanding 86 (2)(2002) 79 – 95.

[5] P. L. Rosin, E. Ioannidis, Evaluation of global image thresholding for change detection, Pattern Recog-nition Letters 24 (14) (2003) 2345 – 2356.

[6] G. Vegas-Sanchez-Ferrero, S. Aja-Fernandez, C. Palencia, M. Martin-Fernandez, A generalized gammamixture model for ultrasonic tissue characterization, Computational and mathematical methods inmedicine 2012.

[7] N. Otsu, A threshold selection method from gray-scale histogram, IEEE Trans. on Sys., Man and Cyb.9 (1) (1979) 62–66.

[8] T. Kurita, N. Otsu, N. Abdelmalek, Maximum likelihood thresholding based on population mixturemodels, Pattern Recognition 25 (10) (1992) 1231–1240.

[9] M. Sezgin, B. Sankur, Survey over image thresholding techniques and quantitative performance evalu-ation, Journal of Electronic Imaging 13 (1) (2004) 146–165.

[10] H. Wu, J. Barba, J. Gil, Iterative thresholding for segmentation of cells from noisy images, Journal ofMicroscopy 197 (3) (2000) 296–304.

[11] C. Leung, F. Lam, Maximum a posteriori spatial probability segmentation, Vision, Image and SignalProcessing, IEE Proceedings- 144 (3) (1997) 161–167.

[12] P. K. Sahoo, S. Soltani, A. K. C. Wong, A survey of thresholding techniques, Computer Vision, Graphics,and Image Processing 41 (2) (1988) 233 – 260.

[13] R. C. Gonzalez, R. E. Woods, Digital Image Processing, Addision Wesley, 1992.

12

[14] L. Mahmoudi, A. El Zaart, A survey of entropy image thresholding techniques, in: Advances in Com-putational Tools for Engineering Applications (ACTEA), 2012 2nd International Conference on, IEEE,2012, pp. 204–209.

[15] S. U. Lee, S. Yoon Chung, R. H. Park, A comparative performance study of several global thresholdingtechniques for segmentation, Computer Vision, Graphics, and Image Processing 52 (2) (1990) 171–190.

[16] F. R. D. Velasco, Thresholding using the isodata clustering algorithm, IEEE Trans. on Sys., Man andCyb. SMC-10 (1980) 771–774.

[17] H. Lee, R. H. Park, Comments on an optimal threshold schemefor image segmentation, IEEE Trans. onSys., Man and Cyb. SMC-20 (1990) 741–742.

[18] W. Cai, S. Chen, D. Zhang, Fast and robust fuzzy c-means clustering algorithms incorporating localinformation for image segmentation, Pattern Recognition 40 (3) (2007) 825–838.

[19] Y. Yong, Z. Chongxun, L. Pan, A novel fuzzy c-means clustering algorithm for image thresholding,Measurement Science Review 4 (1) (2004) 11–19.

[20] A. Masood, Al-Jumaily, A. Ali, Fuzzy c mean thresholding based level set for automated segmentationof skin lesions, Journal of Signal and Information Processing 4 (2013) 66.

[21] B. N. Li, C. K. Chui, S. Chang, S. H. Ong, Integrating spatial fuzzy clustering with level set methodsfor automated medical image segmentation, Computers in Biology and Medicine 41 (1) (2011) 1–10.

[22] T. Pun, A new method for gray-level picture threshold using theentropy of the histogram, Signal Proc-cessing 23 (1980) 223–237.

[23] T. Pun, Entropic thresholding: A new approach, Computer Graphics and Image Processing 16 (1981)210–239.

[24] J. N. Kapur, P. K. Sahoo, A. K. C. Wong, A new method for gray-level picture thresholding using theentropy of the histogram, Graph. Models Image Process. 29 (1985) 273–285.

[25] L. Hertz, R. W. Schafer, Multilevel thresholding using edge matching, Comput. Vis. Graph. ImageProcess. 44 (1988) 279–295.

[26] L. OGorman, Binarization and multithresholding of document images using connectivity, Graph. ModelsImage Process. 56 (1994) 494–506.

[27] S. K. Pal, R. A. King, A. A. Hashim, Automatic gray level thresholding through index of fuzziness andentropy, Pattern Recogn. Lett. 1 (1980) 141–146.

[28] C. A. Murthy, S. K. Pal, Fuzzy thresholding: A mathematical framework, bound functions and weightedmoving average technique, Pattern Recogn. Lett. 11 (1990) 197–206.

[29] K. Ramar, S. Arunigam, S. N. Sivanandam, L. Ganesan, , D. Manimegalai, Quantitative fuzzy measuresfor threshold selection, Pattern Recogn. Lett. 21 (2000) 1–7.

[30] O. J. Tobias, R. Seara, Image segmentation by histogram thresholding using fuzzy sets, IEEE Trans.Image Proccess. 11 (12) (2002) 1457–1465.

[31] M. Forero, Fuzzy thresholding and histogram analysis, in: M. Nachtegael, D. Van der Weken, D. Van deVille, E. Kerre (Eds.), Fuzzy Filters for Image Processing, Springer, Berlin, 2003, pp. 129–152.

[32] A. Z. Arifin, A. F. Heddyanna, H. Studiawan, Ultrafuzziness optimization based on type ii fuzzy sets forimage thresholding, ITB Journal of Information and Communication Technology 4 (2) (2010) 79–94.

13

[33] H. R. Tizhoosh, Image thresholding using type II fuzzy sets, Pattern recognition 38 (12) (2005) 2363–2372.

[34] J. M. Mendel, Fuzzy logic systems for engineering: A tutorial, Proceedings of the IEEE 83 (3) (1995)345–377.

[35] H. R. Berenji, Fuzzy Logic Controllers: An Introduction to Fuzzy Logic Applications and IntelligentSystems, Kluwer Academic Publisher, Boston, Mass, 1992.

[36] P. Szczepaniak, G. L. P.J, J. Kacprzyk (Eds.), Fuzzy systems in medicine, Physica-Verlag, New York,2000.

[37] E. Kerre, M. Nachtegael (Eds.), Fuzzy techniques in image processing, Physica-Verlag, New York, 2000.

[38] D. Van De Ville, M. Nachtegael, D. Van der Weken, E. Kerre, W. Philips, I. Lemahieu, Noise reductionby fuzzy image filtering, IEEE Trans. Fuzzy Sys. 11 (4) (2003) 429–436.

[39] L. Garmendia, The evolution of the concept of fuzzy measure, in: Intelligent Data Mining, Springer,2005, pp. 185–200.

[40] M. Larbani, C.-Y. Huang, G.-H. Tzeng, A novel method for fuzzy measure identification, InternationalJournal of Fuzzy Systems 13 (1) (2011) 24–34.

[41] C. Jawahar, P. K. Biswas, A. Ray, Investigations on fuzzy thresholding based on fuzzy clustering,Pattern Recognition 30 (10) (1997) 1605–1613.

[42] N. Ramesh, J.-H. Yoo, I. Sethi, Thresholding based on histogram approximation, IEE Proceedings-Vision, Image and Signal Processing 142 (5) (1995) 271–279.

[43] L.-K. Huang, M.-J. J. Wang, Image thresholding by minimizing the measures of fuzziness, PatternRecognition 28 (1) (1995) 41–51.

[44] M. S. Prasad, T. Divakar, B. S. Rao, N. Raju, Unsupervised image thresholding using fuzzy measures,International Journal of Computer Applications 27 (2) (2011) 32–41.

[45] S. K. Pal, A. Rosenfeld, Image enhancement and thresholding by optimization of fuzzy compactness,Pattern Recogn. Lett. 7 (1988) 77–86.

[46] S. Di Zenzo, L. Cinque, S. Levialdi, Image thresholding using fuzzy entropies, Systems, Man, andCybernetics, Part B: Cybernetics, IEEE Transactions on 28 (1) (1998) 15–23.

[47] B. Akay, A study on particle swarm optimization and artificial bee colony algorithms for multilevelthresholding, Applied Soft Computing 13 (6) (2013) 3066 – 3091.

[48] P. Sathya, R. Kayalvizhi, Modified bacterial foraging algorithm based multilevel thresholding for imagesegmentation, Engineering Applications of Artificial Intelligence 24 (4) (2011) 595 – 615.

[49] M.-H. Horng, Multilevel minimum cross entropy threshold selection based on the honey bee matingoptimization, Expert Systems with Applications 37 (6) (2010) 4580 – 4592.

[50] J. Bezdek, FCM: The fuzzy c-means clustering algorithm, Computers and Geosciences 10 (1984) 191–203.

[51] D. Drumheller, General expressions for Rician density and distribution functions, IEEE Trans. onAerospace and Electronic Systems 29 (2) (1993) 580–588.

[52] H. Gudbjartsson, S. Patz, The Rician distribution of noisy MRI data, Magn. Reson. Med. 34 (6) (1995)910–914.

14

[53] G. Vegas-Sanchez-Ferrero, D. Martın-Martinez, S. Aja-Fernandez, C. Palencia, On the influence of in-terpolation on probabilistic models for ultrasonic images, in: Proc, of the ISBI, Rotterdam, Netherlands,2010, pp. 292–295.

[54] X. Zeng, M. Singh, Approximation theory of fuzzy systems- SISO case, IEEE Trans. Fuzzy Sys. 2 (2)(1994) 162–176.

[55] S. Aja-Fernandez, C. Alberola-Lopez, G. Cybenko, A fuzzy MHT algorithm applied to text-basedinformation tracking, IEEE Trans. Fuzzy Sys. 10 (3) (2002) 360–374.

[56] B. Devi, V. Sarma, Estimation of fuzzy memberships from histograms, Information Sciences 35 (1)(1985) 43 – 59.

[57] A. Gillet, L. Macaire, C. Botte-Lecocq, J.-G. Postaire, Color image segmentation by analysis of 3dhistogram with fuzzy morphological filters, in: M. Nachtegael, D. Van der Weken, E. Kerre, D. VanDe Ville (Eds.), Fuzzy Filters for Image Processing, Vol. 122 of Studies in Fuzziness and Soft Computing,Springer Berlin Heidelberg, 2003, pp. 153–177.

[58] C.-C. Yang, N. Bose, Generating fuzzy membership function with self-organizing feature map, PatternRecognition Letters 27 (5) (2006) 356 – 365.

[59] F. Russo, {FIRE} operators for image processing, Fuzzy Sets and Systems 103 (2) (1999) 265 – 275.

[60] H. Tizhoosh, Fast fuzzy edge detection, in: Fuzzy Information Processing Society, 2002. Proceedings.NAFIPS. 2002 Annual Meeting of the North American, 2002, pp. 239–242.

[61] K.-S. Chuang, H.-L. Tzeng, S. Chen, J. Wu, T.-J. Chen, Fuzzy c-means clustering with spatial infor-mation for image segmentation, Computerized Medical Imaging and Graphics 30 (1) (2006) 9 – 15.

[62] D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and itsapplication to evaluating segmentation algorithms and measuring ecological statistics, in: Proc. 8thInt’l Conf. Computer Vision, Vol. 2, 2001, pp. 416–423.

[63] W. M. Rand, Objective criteria for the evaluation of clustering methods, Journal of the AmericanStatistical Association 66 (1971) 846–850.

[64] M. Meila, Comparing clusterings: An axiomatic view, in: Proc. of the 22nd International Conferenceon Machine Learning, New York, NY, USA, 2005, pp. 577–584.

[65] P. Arbelaez, M. Maire, C. Fowlkes, J. Malik, Contour detection and hierarchical image segmentation,IEEE Trans. Pattern Anal. Mach. Intell. 33 (5) (2011) 898–916.

15

(a) Cell (b) Cell (additive noise) (c) Cell (mult. noise)

(d) Ball (e) Ball (σn = 30) (f) Ball (σn = 80)

(g) Liver (h) Radiograph

Figure 4: Original images used for the experiments.

16

Figure 5: Influence of different centroid extraction methods over the final segmentation. (a,d) PTS MF after maxima search;(b,e) Gaussian MF after Gaussian fitting; (c,f) PTS MF after clustering. Top row (a,b,c) image smoothed with Gaussian kernel.Lower row (d,e,f) no smoothing.

MedAg AvAg IterAg AbsMax

Figure 6: Comparison of aggregation methods. Top row: original. Middle: additive Gaussian noise (σ = 10). Bottom: additiveGaussian noise (σ = 20)

17

MedAg AvAg IterAg AbsMax

Figure 7: Comparison of aggregation methods.

18

σ = 30 σ = 80 σ = 30 σ = 80O

tsu

S-F

CM

ITS

Max

FT

-II

Med

Ag

Ots

uN

AvA

g

FC

M

Iter

Ag

MA

SP

Figure 8: Comparison of thresholding methods for the phantom corrupted by additive Gaussian noise. Two different levels ofnoise are considered, σ = 30 and σ = 80.

19

Original cell Additive noise Multip. noise Original liver

Ots

uIT

SF

T-I

IO

tsu

-NF

CM

MA

SP

S-F

CM

Max

Med

Ag

AvA

gIt

erA

g

Figure 9: Comparison of thresholding methods for the cell and liver image.

20