jasc: journal of applied science and computations issn no ... · vijay shrinath patil #1, pramod...

Modified local tetra pattern for image retrieval and indexing Vijay Shrinath Patil#1, Pramod Jagan Deore#2

#Department of Electronics and Telecommunication,

R. C. Patel Institute of Technology, Shirpur-425405, (India)

[email protected] [email protected]

Abstract— Representation of image contents using low level features is the most crucial step for content based image retrieval. This

paper proposes a novel feature using modified local tetra pattern for content based image retrieval. The proposed method describes

the new feature by encoding the direction and magnitude of the reference pixel with its neighbouring pixels. For defining the direction

patterns, first order derivatives along the 45o and 135o are considered. Also the magnitude patterns along these directions are

computed. The generated direction and magnitude patterns are converted into rotational invariant features and concatenated with

each other to form Modified Local Tetra Pattern (MLTrP). The performance of the proposed approach is evaluated on two different

benchmark databases. The average precision of the proposed system is improved from 72% to 78% and 70% to 82% as compared to

local binary pattern on Wang database and Brodatz database respectively. The experimental evaluation shows that the proposed

approach outperforms the other existing approaches.

Keywords— Content based image retrieval, local binary pattern, local derivative pattern, local tetra pattern, semantic gap.

I. INTRODUCTION

A. Motivation

There is a vast demand for CBIR systems in various applications such as crime prevention, remote sensing, historical

research, home entertainment etc. Because of the requirements of retrieving the desired images in these applications, many

content based image retrieval (CBIR) systems have been developed. The effectiveness of the CBIR system depends upon which

low level features are used for image representation. So, in the last two decades many researcher communities are being

attracted towards designing the new techniques for image representation and indexing [1, 2, 3]. A detailed survey about these

CBIR systems is available in [4, 5]. The retrieval in CBIR is performed by matching the features of a query image with features

of the images in the database. The image features can be derived from spatial data [6] or from transformed images [7]. The

spatial features derived from the image pixel values are mostly used in many real-time applications. From transformed images,

features are created using different image transforms. Texture feature is used as a prominent feature in many applications such

as pattern analysis, remote sensing, face recognition, and biomedical image analysis. The major problems with texture analysis

are variation in scale, orientation, uneven illumination, and computational complexity. Most of the recent work on texture

analysis is focused on these problems. Moghaddam proposed wavelet correlogram [8] for image indexing and retrieval using

texture as the dominant feature. Occurrence matrices [9], feature anisotropy [10] are the first few approaches that addressed the

rotation invariant feature description. The computational complexity of most of the texture analysis methods is too high. To

address the computational complexity of the texture analysis, Ojala [11] proposed the Local Binary Pattern (LBP) feature,

which measures the spatial structure of the local image texture. The success of LBP leads to its variants such as dominant LBPs

[12], completed LBPS [13]. Also, some researchers proved that using the fusion of various features is more effective than using

a single feature for image retrieval.

RELATED WORK

In last decade, most of the research work is focused on finding the less complex spatial techniques to represent the image

local patterns. LBP feature measures the spatial structure of the image texture. LBP converts the relationship of a pixel with its

surrounding pixels by using local derivatives. The success of LBP is a big start to represent image texture using the local

patterns such as Local Derivative Patterns (LDP) [14], Local Ternary Patterns (LTP) [15], and Local Tetra Patterns (LTrP) [16].

Because of the advantages of LBPs such as less complexity and invariance to illumination changes, LBP became more popular

in texture classification and face recognition.

Pujol and Garcia [17] used LBP generated patterns for face recognition. In this approach, the face image is divided into four

regions using four different masks. Then LBP histograms are calculated for each region and concatenated with each other. From

this combined histogram the important information required for face recognition is obtained by means of data mining tools.

JASC: Journal of Applied Science and Computations

Volume VI, Issue IV, April/2019

ISSN NO: 1076-5131

Page No:690

The proposed system achieved good recognition rate with less dimension of the feature vector, but it is more prone to drastic

changes in facial expression and variations in pose. These two problems are overcome by Muquneel and Holambe [18] by

proposing a local binary pattern based on directional wavelet transform for expression and pose-invariant face recognition. In

this method, quad tree partitioning is used to form the sub-bands for the directional wavelet transform. Then LBP histogram

features are extracted from these sub-bands to obtain the local descriptive features. Based on the concept that the appearance of

the interested region can be well described by the local features, Heikkila [19] used LBP to characterize the interested region.

The authors proposed Center Symmetric LBP [CS-LBP], which possesses less computational complexity as compared to Scale-

Invariant Feature Transform [SIFT]. The experimental evaluation shows the CS-LBP outperforms the SIFT descriptor and is

also tolerant to illumination changes.

An extended version of LBP called a local derivative pattern is proposed by Zhang et al. [14]. In this approach, they extracted

more detailed information about local pattern using first order derivatives along four specific directions. The use of LBP for

content based image retrieval is reported in [20]. In this approach author have addressed the problem of appearance variation

caused due to illumination, pose, facial expression, and aging by proposing a robust LBP for image retrieval. The robust LBP

operator forms the feature vector by combining the sign and magnitude of local difference operator. The robustness is confirmed

by conducting experiments on different image databases under different lighting and noise conditions. Most of the recent work

proposed feature fusion algorithms for CBIR. Giveki et al. [21] proposed a new feature descriptor for CBIR by combining SIFT

and LDP. The proposed descriptor models the appearance and shape of the local object or scene by computing the distribution

of edge directions, intensity gradients or texture features. Fadaei [22] proposed a multilevel coding approach instead of binary

coding as in LBP. The author claims that the multilevel coding preserves more information in features used to describe the

image content. The proposed approach is based on the pixel value differences along a line and weighted combination of these

differences. Singn et al. [23] extended the use of LBP for color images. In the proposed approach the feature vector is the

fusion of the three components. The first component represents the spatial relationship among color pixel. The second

component and third component is the LBP and color histogram of the H component of the HSI color space respectively.

Inspired from approach proposed by Murala et al. [16], we have proposed an extended version of LTrP called as Modified Local

Tetra Pattern (MLTrP). The LTrP proposed by Murala considers the local derivatives along 0o and 90o directions, in the

proposed approach we have considered the local derivative along 45o and 135o.

The rest of the paper is organized as follows. Section-III discusses the proposed work. The experimental evaluation is given

in section IV and the paper is concluded in section V.

II. MODIFIED LOCAL DERIVATIVE PATTERNS (MLTRPS)

In this section first different local patterns are discusses and then modified third order local tetra pattern is discussed.

A. Local Binary Pattern (LBP)

LBP is the gray scale invariant texture representation, which is calculated using the first order derivative of a pixel with its

neighboring pixels. LBP encodes texture by applying the thresholding function on the result of first order derivative. LBP is

introduced by ojala [11] and has shown as an excellent performance in texture classification [24], face recognition [25] and

pattern analysis.

LBP is defined by equation (1)

(1)

Where gn is the neighboring pixel of center pixel gc, equally spaced on the circle with radius R. The thresholding function

T is define as

B. Local Derivative Pattern (LDP)

To obtain more detailed discriminative features from an image, Zhang[14] proposed local derivative pattern (LDP). As LBP

consider first order derivative in all directions, the LDP encodes higher order derivative information in the 00, 450, 900 and 1350

directions.

To calculate the nth order derivative, first they calculated n-1th order derivative that is . Then by

using this order derivative, the nth order derivative is calculated using equation (2)



ISSN NO: 1076-5131

Page No:691

(2)

C. Local Ternary Pattern (LTrP)

An extended version of LBP called LTrP is proposed by Murala et al. [16]. The authors claim that the LTrP is more efficient

than LBP and LDP. The LTrP considers the derivatives along the horizontal and vertical directions, in four different directions.

To compute the LTrP first the different direction numbers (D= 1, 2, 3, 4) are computed using the gradient along the horizontal

and vertical directions. Then from these direction numbers, 3 different patterns are formed for each of the 4 different directions.

So total 4 X 3 = 12 patterns are generated. Thus the whole image is converted into directional values.

III. PROPOSED WORK

Fig. 1 Block diagram of proposed image retrieval system

The LTrPs proposed in [16] is further modified to propose the modified local tetra pattern called MLTrP. The MLTrPs

describes the texture pattern using the direction of the center pixel and its neighboring pixels. Figure 1 shows the overall

architecture of the proposed approach.

Let I be the image, to be used for the texture analysis. The first order derivative along 450 and 1350directions denoted

by D’θ. Let gc denote the center pixel and and denotes the diagonal neighbors of along 450 and 1350 direction

respectively. Then the first order derivative is defined as given in Eq. (3)

(3)

Using above defined derivatives the direction of center pixel is decided as in Eq. (4)



ISSN NO: 1076-5131

Page No:692

(4)

Thus the center pixel can have four different direction, using gradient along 45oand 135o directions. Using these directions, the

second order derivative (D2) is defined as in equation (5)

------ (5)

(6)

Equation (5) and (6) produces 8-bit tetra pattern for each center pixel. This 8-bit pattern is then converted into 3 binary patterns

using the direction values in the tetra pattern. Figure 2 shows the example of MLTrP. Similarly, the binary patterns are formed

for directions 2, 3 & 4. Thus for every center pixel 12 patterns are formed.

As said in [26] if we combine the sign component and the magnitude component, the texture classification accuracy

increases. Thus 1-magnitude patterns along 450and 1350 directions is computed along with the 12 direction patterns, for each

center pixel.

With p neighbours of a center pixel gives 2p combinations of patterns are possible. This gives the feature vector of

length 2p . Thus, to reduce the computational cost of the feature vector, only uniform patterns are considered. The uniform

patterns are those, having a limited number of discontinuities in circular representation of the pattern. In this approach, the

patterns with 2 discontinuities are considered as the uniform pattern. Also the generated patterns are not rotation invariant. Thus

the uniform patterns are also converted into the rotation invariant. The example of generating the rotation invariant pattern is

given in table 1. In the given example, the rotation invariant pattern for 00011001 is 00011001, as the minimum decimal value

among all the circularly shifted decimal values is 25.



ISSN NO: 1076-5131

Page No:693

Fig. 2 Example to obtain MLTrP and magnitude pattern. For a center pixel 12 direction patterns and 1 magnitude pattern is generated.

TABLE I

ILLUSTRATION OF ROTATION INVARIANT PATTERN

Iteration Pattern “1” shifted pattern Decimal number

0 00011001 00011001 25

1 10001100 140

2 01000110 70

3 00100011 35

4 10010001 145

5 11001000 200

6 01100100 100

7 00110010 50

8 00011001 25

In the last step histograms are formed for each pattern to form the feature vector. Thus, MLTrPs encode the relationship of a

center pixel and its neighbouring pixels based on the neighbours along 45° and 135° directions. Whereas, LTrPs encodes the

relationship based on the directions of the center pixel and neighbours along 00 and 900 directions.

IV. EXPERIMENTAL RESULTS

For experimental evaluation of the proposed system two different benchmark databases are used. Precision and recall are the

commonly used performance retrieval metrics [27, 28], which are defined as in equation (7) and (8) respectively.

(7)

R (8)

Where is the number of relevant images retrieved from top N retrieved images and M is the total number of relevant images

present in the database.



ISSN NO: 1076-5131

Page No:694

Here relevant images mean the images that belong to the class of the query image. In experimental evaluation each image is

considered as the query image, and average precision and recall values are computed by considering different values of N.

As mentioned in [29] the performance of a retrieval system is strongly depends on similarity metric along with effective feature

descriptors. As the approach proposed in this paper is histogram based, the extended canbera distance metric proposed in [29] is

used. Let represents the feature vector of the image in the database and

represents the feature vector of the query image. Then the extended canbera distance metric is defined as in Equation (9)

Where and

The experimental results of the proposed system are compared with following three approaches present in the literature.

• LBP: Local Binary Pattern proposed by T. Ojala et al. [11]

• LDP : Local derivative patterns proposed by B. Zhang et al. [14]

• LTrP : Local Tetra Patterns proposed by S. Murala et al. [16]

A. Experiment-1

For experiment-1 Wang database [31] is used. This database is consist of 1000 images, divided into 10 different categories

such as Africa, Beaches, Building, Bus, Dinosaur, Elephant, Flower, Horses, Mountain and Food . All the images in database

are of size 256 × 384 or 384 × 256. Figure 3 shows the retrieval examples for different query images. Figure 4 shows the

precision and recall curves for Wang database. The computed precision and recall values are compared with LBP, LDP and

LTrP. The average precision of the proposed system is improved from 72% to 78% and 76% to 78% as compared to LBP and

LTrP respectively, for Wang database. This graph illustrates the retrieval performance of the MLTrP with other existing

methods as a function of the total number of matched images. From this database it is clear that the proposed approach

outperforms as compared to other existing approaches.

Fig. 3 Retrieval results for Wang database

(9)



ISSN NO: 1076-5131

Page No:695

Fig. 4 (a) Precision and (b) Recall curves for Wang database.

B. Experiment-2

Experiment-2 is performed on Brodatz texture database [32]. This database is created by dividing each of 112 texture images

into 25 sub images having size 128 X 128. Thus, the total images in the database are 2700. Figure 5 shows the retrieval

examples for different query images. Figure 6 shows the precision and recall curves for Brodatz database. The computed

precision and recall values are compared with LBP, LDP and LTrP. This graph illustrates the retrieval performance of the

MLTrP as compared with other existing methods as a function of the total number of matched images. The average precision of

the proposed system is improved from 70% to 82% and 79% to 82% as compared to LBP and LTrP respectively for Brodatz

database. From these results, it is clear that the proposed approach outperforms as compared to other existing approaches.

Fig. 5 Retrieval results for Wang Brodatz database



ISSN NO: 1076-5131

Page No:696

Fig.6 (a) Precision and (b) Recall curves for Brodatz database.

V. Conclusion

This paper proposes a novel feature for content based image retrieval using modified local tetra pattern for content based

image retrieval. The proposed feature encodes the direction and magnitude of the reference pixel with its neighbouring pixels.

For defining the direction patterns, first order derivatives along the 45° and 135° degrees are considered. The generated

direction and magnitude patterns are converted into rotational invariant features and concatenated with each other to form

MLTrPs. The experimental results are evaluated on two different benchmark databases. The performance of the proposed

features is compared with Local Binary pattern, Local Derivative Pattern and Local Tetra Patterns on two different databases.

The work can be further extended by adding derivatives along horizontal and vertical direction along with diagonal and anti-

diagonal directions.

REFERENCES

[1] Ju-xiang Zhou et al., “A new fusion approach for content based image retrieval with color histogram and local directional pattern,”Int. J. Mach. Learn. &

Cyber, vol/issue: 9(4), pp.677–689, 2018.

[2] S. Murala and Q. M. Jonathan Wu, “Local Mesh Patterns Versus Local Binary Patterns: Biomedical Image Indexing and Retrieval,” IEEE Journal of

Biomedical and Health Informatics, vol/issue: 18 (3), pp.929–938, 2014.

[3] S. Vipparthi et al., “Dual directional multi-motif XOR patterns: A new feature descriptor for image indexing and retrieval,” Optik – International Journal

for Light and Electron Optics, vol/issue: 126 (15-16), pp.1467-1473, 2015.

[4] Liu Y et al., “A survey of content based image retrieval with high-level semantics,” Pattern Recognition, vol/issue: 40(1), pp. 262–282, 2007.

[5] Smeulders AWM et al., “Content based image retrieval at the end of the early years,” IEEE Trans Pattern Anal Mach Intell, vol/issue: 22(12), pp.1349–

1380, 2000.

[6] J. Ahmad et al., “Multi-scale local structure patterns histogram for describing visual contents in social image retrieval systems,” Multimed Tools Appl,

vol/issue: 75(20), pp.12669–12692. 2016.

[7] Belkasim S, Hong X, Basir O (2007) Content based image retrieval using discrete wavelet transform”. Int J Pattern Recognition Artificial

Intelligence18(1):19–32

[8] H. A. Moghaddam, T. T. Khajoie, and A. H. Rouhi, “A new algorithm for image indexing and retrieval using wavelet correlogram,” in Proc. ICIP, 2003,

pp. III-497–III-500.

[9] L.S. Davis et al., “Texture Analysis Using Generalized Co-occurrence Matrices”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol/issue: 1(3),

pp. 251-259, 1979.

[10] D. Chetverikov, “Experiments in the Rotation-Invariant Texture Discrimination Using Anisotropy Features,” Proc. Sixth Int'l Conf. Pattern Recognition,

pp. 1071-1073, 1982.



ISSN NO: 1076-5131

Page No:697

[11] Ojala T et al., “Multiresolution grayscale and rotation invariant texture classification with local binary patterns,” IEEE Trans Pattern Anal Mach Intell,

vol/issue: 24(7), pp.971–987, 2002.

[12] S. Liao et al., “Dominant local binary patterns for texture classification,” IEEE Trans. Image Process., vol/issue: 18(5), pp. 1107–1118, 2009.

[13] Z. Guo et al., “A completed modeling of local binary pattern operator for texture classification,” IEEE Trans. Image Process., vol/issue: 19(6), pp. 1657–

1663, 2010.

[14] B. Zhang et al., “Local derivative pattern versus local binary pattern: Face recognition with higher-order local pattern descriptor,” IEEE Trans. Image

Process., vol/issue: 19(2), pp. 533–544, 2010.

[15] X. Tan and B. Triggs, “Enhanced local texture feature sets for face recognition under difficult lighting conditions,” IEEE Trans. Image Process.,

vol/issue: 19(6), pp. 1635–1650, 2010.

[16] S. Murala et al., Local Tetra Patterns: A New Feature Descriptor for Content Based Image Retrieval,” IEEE Transactions on Image Processing, vol/issue:

21 (5), 2874-2886, 2012.

[17] F. Pujol and Juan C. Garcia, “Computing the Principal Local Binary Patterns for face recognition using data mining tools,” Expert Systems with

Applications, vol/issue: 39(8), pp.7165 – 7172, 2012.

[18] M.A. Muqeet et al., “Local binary patterns based on directional wavelet transform for expression and pose-invariant face recognition,” Applied

Computing and Informatics, (Article in press), 2017.

[19] Marko Heikkila et al., “Description of interest regions with local binary patterns,” Pattern Recognition, vol/issue: 42(3), pp.425-436, 2009.

[20] S. Murala et al., “Expert content-based image retrieval system using robust local patterns,” Journal of Visual Communication and Image Representation,

vol/issue: 25(6), pp.1324-1334, 2014.

[21] D. Giveki et al., “A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern,”

Optik, vol/issue: 131, pp. 242-254, 2017.

[22] S. Fadaei et al., “Local derivative radial patterns: A new texture descriptor for content-based image retrieval,” Signal Processing, vol/issue: 137, pp. 274-

286, 2017.

[23] C. Singh et al., “Color texture description with novel local binary patterns for effective image retrieval,” Pattern Recognition, vol/issue: 76, pp.50-68.,

2018.

[24] A. Ahmadian and A. Mostafa, “An efficient texture classification algorithm using Gabor wavelet,” in: 25th Annual international conf. of the IEEE

EMBS, Cancun, Mexico, pp. 930–933, 2003.

[25] Fabiola Becerra-Riera, Annette Morales-González, Heydi Méndez-Vázquez, Facial marks for improving face recognition, Pattern Recognition Letters,

Volume 113, Pages 3-9,2018.

[26] Z. Guo, L. Zhang, and D. Zhang, “Rotation invariant texture classification using LBP variance with global matching,” Pattern Recogn., vol. 43, no. 3, pp.

706–719, Mar. 2010.

[27] H. Muller et al., “Performance evaluation in content-based image retrieval: overview and proposals,” Pattern Recognition Letters, vol/issue: 22 (5), pp.

593–601, 2001.

[28] Y. Yang, “An evaluation of statistical approaches to text categorization,” Information Retrieval, vol/issue: 1 (1–2), pp. 69–90, 1999.

[29] Guang-Hai Liu and Jing-Yu Yang, “Content-based image retrieval using color difference histogram, Pattern Recognition,” vol/issue: 46(1), pp.188-198,

2013.

[30] Wang J.A., “SIMPLIcity: semantics-sensitive integrated matching for picture libraries,” IEEE Trans. Pattern Anal. Mach. Intell. Vol/issue: 23(9),

pp.947–963, 2001.

[31] P. Brodatz, “Textures: A Photographic Album for Artists and Designers,” New York: Dover, 1996.



ISSN NO: 1076-5131

Page No:698

jasc: journal of applied science and computations issn no ... · vijay shrinath patil #1, pramod...

Documents