project report new 03

8/6/2019 Project Report New 03

1/58

1INTRODUCTION

Science is wonderfully equipped to answer the question "How?" but it

gets terribly confused when you ask the question "Why?"

-Erwin Chargaff

CHAPTER 1

INTRODUCTION

SAE KONDHWA, ELECTRONICS AND TELECOMMUNICATION ENGINEERING 2009-2010

1
http://www.brainyquote.com/quotes/quotes/e/erwincharg111216.htmlhttp://www.brainyquote.com/quotes/quotes/e/erwincharg111216.html


2/58

As processors become increasingly powerful, and memories become increasingly

cheaper, the deployment of large image databases for a variety of applications have now

become realizable. Databases of art works, satellite and medical imagery have been

attracting more and more users in various professional fields for example, geography,

medicine, architecture, advertising, design, fashion, and publishing. Effectively and

efficiently accessing desired images from large and varied image databases is now a

necessity.

1.1 DEFINITION

CBIR orContent Based Image Retrievalis the retrieval of images based on visual

features such as colour, texture and shape [17]. Reasons for its development are that in

many large image databases, traditional methods of image indexing have proven to be

insufficient, laborious, and extremely time consuming. These old methods of image

indexing, ranging from storing an image in the database and associating it with a keyword

or number, to associating it with a categorized description, have become obsolete. This is

not CBIR. In CBIR, each image that is stored in the database has its features extracted and

compared to the features of the query image. It involves two steps [17]:

Feature Extraction: The first step in the process is extracting image features to a

distinguishable extent.

Matching: The second step involves matching these features to yield a result that

is visually similar.

1.1.1 Applications of CBIR

Examples of CBIR applications are [17]:

Crime prevention: Automatic face recognition systems, used by police forces.


2


3/58

Security Check: Finger print or retina scanning for access privileges.

Medical Diagnosis: Using CBIR in a medical database of medical images to aid

diagnosis by identifying similar past cases.

Intellectual Property: Trademark image registration, where a new candidate

mark is compared with existing marks to ensure no risk of confusing property

ownership.

1.1.2 CBIR Systems

Several CBIR systems currently exist, and are being constantly developed. Few

Examples are:

QBIC or Query By Image Content was developed by IBM, Almaden Research

Centre, to allow users to graphically pose and refine queries based on multiple

visual properties such as colour, texture and shape [16]. It supports queries based

on input images, user-constructed sketches, and selected colour and texture

patterns [16].

VIR Image Engine by Virage Inc., like QBIC, enables image retrieval based on

primitive attributes such as colour, texture and structure. It examines the pixels in

the image and performs an analysis process, deriving image characterization

features [16]. VisualSEEK and WebSEEK were developed by the Department of Electrical

Engineering, Columbia University. Both these systems support colour and spatial

location matching as well as texture matching [16].

NeTra was developed by the Department of Electrical and Computer Engineering,

University of California. It supports colour, shape, spatial layout and texture

matching, as well as image segmentation [16].

MARS or Multimedia Analysis and Retrieval System was developed by the

Beckman Institute for Advanced Science and Technology, University of Illinois. It

supports colour, spatial layout, texture and shape matching [16].


3


4/58

ViperorVisual Information Processing for Enhanced Retrievalwas developed

at the Computer Vision Group, University of Geneva. It supports colour and

texture matching [16].

1.2 PROBLEM MOTIVATION

Present image search systems(search engines) are mostly text based retrieval

systems. Many times this leads to unwanted results.

Current techniques are based on low level features like colour, texture,shape etc. and

there is a huge semantic gap existing.

Image databases and collections can be enormous in size, containing hundreds,

thousands or even millions of images. The conventional method of image retrieval is

searching for a keyword that would match the descriptive keyword assigned to the image

by a human categorizer . Currently under development, even though several systems exist,

is the retrieval of images based on their content, called Content Based Image Retrieval,

CBIR. While computationally expensive, the results are far more accurate than

conventional image indexing. Hence, there exists a tradeoff between accuracy and

computational cost. This tradeoff decreases as more efficient algorithms are utilized andincreased computational power becomes inexpensive.


4


5/58

1.3 PROBLEM STATEMENT

The problem involves entering an image as a query into a software application that

is designed to employ CBIR techniques in extracting visual properties, and

matching them. This is done to retrieve images in the database that are visually

similar to the query image.


5


6/58

2 LITERATURESURVEY

A mans feet should be planted in his country, but his eyes should survey

the world

-George Santayana


6


7/58

CHAPTER 2

LITERATURE SURVEY

There are many features of an image that can be extracted from the image itself e.g.

colour,texture,shape etc. Some of the features can be explained as follows:

2.1 Colour

2.1.1 Definition

One of the most important features that make possible the recognition of images

by humans is colour. Colour is a property that depends on the reflection of light to the eye

and the processing of that information in the brain[15]. We use colour everyday to tell the

difference between objects, places, and the time of day[18] . Usually colours are defined

in three dimensional colour spaces. These could either be RGB (Red, Green, and Blue),

HSV(Hue, Saturation, and Value) orHSB (Hue, Saturation, and Brightness). The last two

are dependent on the human perception of hue, saturation, and brightness[18].

Most image formats such as JPEG, BMP, GIF, use the RGB colour space to store

information [18]. The RGB colour space is defined as a unit cube with red, green, and

blue axes. Thus, a vector with three co-ordinates represents the colour in this space. When

all three coordinates are set to zero the colour perceived is black. When all three

coordinates are set to 1 the colour perceived is white [18]. The other colour spaces operate

in a similar fashion but with a different perception.

2.1.2 Methods of Representation

The main method of representing colour information of images in CBIR systems is

through colour histograms. A colour histogram is a type of bar graph, where each bar

represents a particular colour of the colour space being used. In MATLAB for example

you can get a colour histogram of an image in the RGB or HSV colour space. The bars in

a colour histogram are referred to as bins and they represent the x-axis [14][13]. The

number of bins depends on the number of colours there are in an image. The y-axis

denotes the number of pixels there are in each bin. In other words how many pixels in an

image are of a particular colour [14].


7


8/58

An example of a colour histogram in the HSV colour space can be seen with the

following image:

Figure 2.1(a): Sample Image (Source: Internet)

Figure 2.1(b): H,S,V components of sample image


8


9/58

As shown in figure we have plotted histograms and image planes corresponding to

H,S,V components of sample image.

Quantization in terms of colour histograms refers to the process of reducing the

number of bins by taking colours that are very similar to each other and putting them in

the same bin. By default the number of bins one can obtain using the histogram function

in MATLAB is 256.

HSV colour space is used for extracting colour features because of following

possible advantages:

HSV colour space is designed to be similar to human colour perception.

In HSV colour space, H and S planes are more important because they represent

colour; V represents value or intensity of grayscale image. Hence we can give less

importance to V plane of the image. This results in greater computational

efficiency i.e. improved time complexity.


9


10/58

2.2 Texture

2.2.1 Definition

Texture is that innate property of all surfaces that describes visual patterns, each

having properties of homogeneity. It contains important information about the structural

arrangement of the surface, such as; clouds, leaves, bricks, fabric, etc. It also describes the

relationship of the surface to the surrounding environment [1]. In short, it is a feature that

describes the distinctive physical composition of a surface.

Texture properties include:

Coarseness

Contrast

Directionality

Line-likeness

Regularity

Roughness

Texture is one of the most important defining features of an image. It is

characterized by the spatial distribution of gray levels in a neighborhood [1][6]. In order

to capture the spatial dependence of gray-level values, which contribute to the perception

of texture, a two-dimensional dependence texture analysis matrix is taken into

consideration. This two-dimensional matrix is obtained by decoding the image file; jpeg,

bmp, etc.


10

(a)

Clouds(b)

Bricks

(c)

Rocks

Figure 2.2: Examples of Textures. (Source: Internet)


11/58


There are three principal approaches used to describe texture; statistical, structural

and spectral

Statistical techniques characterize textures using the statistical properties of the

grey levels of the points/pixels comprising a surface image. Typically, these

properties are computed using: the grey level co-occurrence matrix of the

surface, or the wavelet transformation of the surface.

Structural techniques characterize textures as being composed of simple

primitive structures called texels (or texture elements). These are arranged

regularly on a surface according to some surface arrangement rules.

Spectral techniques are based on properties of the Fourier spectrum and

describe global periodicity of the grey levels of a surface by identifying high-

energy peaks in the Fourier spectrum [7].

For optimum classification purposes, what concern us are the statistical techniques

of characterization. This is because it is these techniques that result in computing texture

properties. The most popular statistical representations of texture are:

Co-occurrence Matrix

Wavelet Packet Decomposition


11


12/58

2.2.2.1 Co-occurrence Matrix

Originally proposed by R.M. Haralick, the co-occurrence matrix representation of

texture features explores the grey level spatial dependence of texture. A mathematical

definition of the co-occurrence matrix is as follows [12]:

- Given a position operatorP(i,j),

- letA be an n x n matrix

- whose element A[i][j] is the number of times that points with grey level

(intensity) g[i] occur, in the position specified by P, relative to points with

grey levelg[j].

- Let Cbe the n x n matrix that is produced by dividingA with the total number

of point pairs that satisfy P. C[i][j]is a measure of the joint probability that a

pair of points satisfyingPwill have valuesg[i],g[j].

- Cis called a co-occurrence matrix defined byP.

-

Examples for the operatorPare: iabovej, or ione position to the right and two below

j, etc. [12]

This can also be illustrated as follows Let tbe a translation, then a co-occurrence matrix

Ctof a region is defined for every grey-level (a, b)by [3] :

C a b c a r d s s t R A s a A s t bt ( , ) { ( , ) | [ ] , [ ] }= + = + =2

..(2.1)

Here, Ct(a, b) is the number of site-couples, denoted by (s, s + t) that are separated by a

translation vectort, with abeing the grey-level ofs, and bbeing the grey-level ofs + t.

For example; with an 8 grey-level image representation and a vector tthat considers only

one neighbour, we would find [3]:


12


13/58

1 2 1 3 4

2 3 1 2 4

2 3 1 2 4

3 3 2 1 1

Figure 2.3: Image example (8 grey-level image)

0 1 2 3 4 5 6 7

0 0 0 0 0 0 0 0 0

1 0 1 2 0 0 0 0 0

2 0 1 0 2 0 0 0 0

3 0 0 1 1 0 0 0 0

4 0 0 0 0 1 0 0 0

5 0 0 0 0 0 0 0 0

6 0 0 0 0 0 0 0 0

7 0 0 0 0 0 0 0 0

Figure 2.4 : Classical Co-occurrence matrix

At first the co-occurrence matrix is constructed, based on the orientation and

distance between image pixels. Then meaningful statistics are extracted from the matrix as

the texture representation [1]. Haralick proposed the following texture features [8]:

1. Angular Second Moment

2. Contrast

3. Correlation

4. Variance

5. Inverse Second Differential Moment

6. Sum Average

7. Sum Variance


13


14/58

8. Sum Entropy

9. Entropy

10. Difference Variance

11. Difference Entropy

12. Measure of Correlation 1

13. Measure of Correlation 2

14. Local Mean

Hence, for each Haralick texture feature, we obtain a co-occurrence matrix. These

co-occurrence matrices represent the spatial distribution and the dependence of the grey

levels within a local area. Each (i,j) th entry in the matrices, represents the probability of

going from one pixel with a grey level of ' i' to another with a grey level of 'j' under a

predefined distance and angle. From these matrices, sets of statistical measures are

computed, called feature vectors [1].


14


15/58

2.2.2.2 Wavelet Packet Decomposition

Textures can be modeled as quasi-periodic patterns with spatial/frequency

representation. The wavelet transform transforms the image into a multi-scale

representation with both spatial and frequency characteristics. This allows for effective

multi-scale image analysis with lower computational cost [4]. According to this

transformation, a function, which can represent an image, a curve, signal etc., can be

described in terms of a coarse level description in addition to others with details that range

from broad to narrow scales [4].

Unlike the usage of sine functions to represent signals in Fourier transforms, in

wavelet transform, we use functions known as wavelets. Wavelets are finite in time, yet

the average value of a wavelet is zero [4]. In a sense, a wavelet is a waveform that is

bounded in both frequency and duration. While the Fourier transform converts a signal

into a continuous series of sine waves, each of which is of constant frequency and

amplitude and of infinite duration, most real-world signals (such as music or images) have

a finite duration and abrupt changes in frequency. This accounts for the efficiency of

wavelet transforms. This is because wavelet transforms convert a signal into a series of

wavelets, which can be stored more efficiently due to finite time, and can be constructed

with rough edges, thereby better approximating real-world signals [9].

Mathematically, we denote a wavelet as [4]

a, b(t) = (2.2)

Where, b is location parameter and a is scaling parameter. For function to

be a Wavelet, it should be time limited

We define Wavelet transform as [4]

W(a, b) = dt .(2.3)


15


16/58

Examples of wavelets are Coiflet, Morlet, Mexican Hat, Haar and Daubechies. Of

these, Haar is the simplest and most widely used, while Daubechies have fractal structures

and are vital for current wavelet applications [4]. These two are outlined below:

Haar Wavelet

The Haar wavelet family is defined as [4]:

1

0 1 t

-1

Figure 2.5 : Haar Wavelet Example [4]

Haar wavelet scaling function is defined as:

1 0


17/58

Daubechies Wavelet

The Daubechies wavelet family is defined as:

Figure 2.6 : Daubechies Wavelet Example (Source: Internet)

Daubechies wavelet scaling function is defined as:

= (2.5)

Where,

Ck=h(k).

Ck is unnormalized coefficient and h(k) is normalized coefficient.


17


18/58

2.3 SHAPE

2.3.1 Definition

Shape may be defined as the characteristic surface configuration of an object; an

outline or contour. It permits an object to be distinguished from its surroundings by its

outline [10]. Shape representations can be generally divided into two categories [11]:

Boundary-based, and

Region-based.

Figure 2.7: Boundary-based & Region-based Shape representations(Source: Internet)

Boundary-based shape representation only uses the outer boundary of the shape.

This is done by describing the considered region using its external characteristics; i.e., the

pixels along the object boundary. Region-based shape representation uses the entire shape

region by describing the considered region using its internal characteristics; i.e., the pixels

contained in that region [11].


18


19/58


For representing shape features mathematically, we have [11]:

Boundary-based:

Polygonal Models, boundary partitioning

Fourier Descriptors

Splines, higher order constructs

Curvature Models

Region-based:

Superquadrics

Fourier Descriptors

Implicit Polynomials

Blum's skeletons

The most successful representations for shape categories are Fourier Descriptors and

Moment Invariants [11]:

The main idea of Fourier Descriptor is to use the Fourier transformed

boundary as the shape feature.

The main idea of Moment invariants is to use region-based moments, which

are invariant to transformations as the shape feature.


19


20/58

3CBIR METHODOLOGY

The whole is equal to the sum of its parts

-Euclid

The whole is greater than the sum of its parts

-Max Wertheimer


20


21/58

CHAPTER 3

CBIR METHODOLOGY

The solution proposed is to extract the primitive features of a query image and

compare them to those of database images. The image features under consideration are

colour, texture and shape. Thus, using matching and comparison algorithms, the colour,

texture and shape features of one image are compared and matched to the corresponding

features of another image. This comparison is performed using colour, texture and shape

distance metrics. In the end, these metrics are performed one after another, so as to

retrieve database images that are similar to the query. The similarity between features is to

be calculated using specific algorithm. For each specific feature there is a specific

algorithm for extraction and another for matching.

Figure 3.1: CBIR Methodology


21

Relevance

Feedback

USE Query

Formation

Visual

Content

Description

Feature

Vectors

Similarity

Comparison

ImageDatabase

VisualContent

Description

FeatureDatabase

Indexing &Retrieval

OUTPUT Retrieval results


22/58

4 IMPLEMENTATION&

RESULTS

Results! Why, man, I have got a lot of results. I know several thousand

things that wont work.

-Thomas A. Edison


22


23/58

CHAPTER 4

IMPLEMENTATION & RESULTS

4.1 COLOUR

Retrieving images based on colour similarity is achieved by computing a colour

histogram for each image that identifies the proportion of pixels within an image holding

specific values (that humans express as colours). Examining images based on the colours

they contain is one of the most widely used techniques because it does not depend on

image size or orientation.

Algorithm Colour Histogram Matching : Feature Matching

Most commercial CBIR systems include colour histogram as one of the features

(e.g., QBIC of IBM).

(4.1)

Where, j denotes the various gray levels

Hist_query is the histogram of query image,

Hist_database is histogram of the database image.

Histdist is error difference or distance metric

The nearest matching database images with the query image has the least distance

metric. The exact match is the one with the zero distance metric.


23


24/58

Figure 4.1 : Flow of implemented image retrieval system

In the figure above, once a query image and a retrieval method are chosen by

users, the rest of whose process is done automatically. However, the histogram data for all

images in database are computed and saved in Database in advance so that only the image

indexes and histogram data can be used to compare the query image with images in

Database.

4.1.1 RESULTS (colour)


24

Compute the histogram

for each plane

Separate H, S, V

Planes

Compare

histo ram

Show retrieved

imagesDatabase

Precalculate &save histogram

each plane (H,S,V)

in advance


25/58

Here is one of the results using colour histogram matching.

QUERY IMAGE:-

Figure 4.2(a):A Query Image(1) (Source: Internet)

RETRIEVED IMAGES:

Figure 4.2(b): Retrieved Images for query image(1)


25



26/58

QUERY IMAGE:-


RETRIEVED IMAGES :-



26


27/58

DISCUSSION: Colour based image retrieval gives intended results for images containing natural

beauty and similar kind of images.

It doesnt produce intended search results for images having specific shape or

structure.

We should go for texture as well as shape features of an image to overcome the

shortcomings of colour based retrieval.


27


28/58

4.2 TEXTURE

There are three principal approaches used to describe texture:

Statistical,

Structural,

Spectral.

For optimum classification purposes, what concern us are the statistical techniques

of characterization. This is because it is these techniques that result in computing texture

properties. The most popular statistical representations of texture are:

Co-occurrence Matrix

Wavelet Packet Decomposition

4.2.1 Co-occurrence Matrix

Co-occurrence matrices are a popular representation for the texture in image. They

contain a count of the number of times that a given feature (e.g. given gray level)

occurs in a particular spatial relation to another given feature. However, because

of the large number of spatial relations that are possible within an image,

heuristic or interactive techniques have usually been employed to select the

relation to use for each problem. In this project we present a statistical approach

for finding those spatial relations that best capture the structure of textures when

the co-occurrence matrix representation is used. These matrices should thus be

well suited for discriminations that are structurally based[7].


28


29/58

We have used graycomatrix ( ) function inMATLAB for finding gray-level co-

occurrence matrix from the image.

glcms = graycomatrix(I,param1,val1,param2,val2,...)

It returns one or more gray-level co-occurrence matrices, depending on the values

of the optional parameter/value pairs.

One of the parameters that can be passed to graycomatrix function is offset.

Because the offset is often expressed as an angle, the following table lists theoffset values that specify common angles, given the pixel distance D.

Angle Offset

0 [0 D]

45 [-D D]

90 [-D 0]

135 [-D -D]

Table 4.1: Offset values for corresponding angles

The figure 4.4 illustrates the array: offset = [0 1; -1 1; -1 0; -1 -1].

Figure 4.4: Representation of

array offset (Source: MATLABHelp)

We have considered different directions for calculating distance between pixel of

interest and its neighbour. We found that 900 having corresponding offset of [-1 0] yields

better retrieval results.


29

900 [-1 0]

00 [0 1]

450 [-1 1]1350 [-1 -1]

ixel-of-interest


30/58

After calculating co-occurrence matrix representation for a gray scale image we

found standard deviation for each column. We have compared this standard deviation

vector with the extracted standard deviation vectors of corresponding co-occurrence

matrices of images in database. Based on the similarity between these standard deviation

vectors similar images are retrieved.

(4.2)

Where,

j= number of image in database

std_coocc_query = standard deviation vector of co-occurrence matrix for

corresponding query image,

std_coocc _database = standard deviation vector of co-occurrence matrix of

corresponding database image.

coocc_dist = error difference or distance metric.



Figure 4.5 : Flow of co-occurrence matrix based image retrieval system


30

Determine standard deviation for

each column of co-occ matrix

Calculate co-occurrence

matrix of query image

Compare standard

deviation vectors

Show retrieved

images

Database

Features

Precalculate &

save standard

deviation vectors

in advance


31/58

In the figure above, once a query image is chosen by users, the rest of process is

done automatically. However, the co-occurrence matrix data for all images in database are

computed and saved in Database in advance so that only the image indexes and co-

occurrence data can be used to compare the query image with images in Database.


31


32/58

4.2.1.1 RESULTS (co-occurrence)

Here is one of the results using co-occurrence matrix matching algorithm.

QUERY IMAGE:-


RETRIEVED IMAGES :-



32


33/58

QUERY IMAGE:-

Figure 4.7(a):A Query Image(4)

RETRIEVED IMAGES :-



33


34/58

4.2.2 Wavelet Packet Decomposition

In wavelet analysis, a signal is split into an approximation and a detail. The

approximation is then itself split into a second-level approximation and detail, and the

process is repeated. For an n-level decomposition, there are n+1 possible ways todecompose or encode the signal.

S = A1 + D1

= A2 + S2 + D1

= A3 + D3 + S2 + D1

Figure 4.8 : Wavelet packet Decomposition

In wavelet packetanalysis, the details as well as the approximations can be split.

This yields more than different ways to encode the signal. This is the wavelet

packet decomposition tree.

Figure 4.9 : 2-D Wavelet packet Decomposition


34

D1A1

AAA3

AA2 DA2

S

AD2 DD2

DAA3 ADA3 DDA3 AAD3 DAD3 ADD3 DDD3

D1

S

A1

A3

A2 S2

D3


35/58

The wavelet packet method is a generalization of wavelet decomposition that

offers a richer range of possibilities for signal analysis. The wavelet decomposition tree is

a part of this complete binary tree.

In the wavelet packet decomposition, each detail coefficient vector is also

decomposed into two parts using the same approach as in approximation vector splitting.

This offers the richest analysis: the complete binary tree is produced in the one-

dimensional case or a quaternary tree in the two-dimensional case.

Two-dimensional DWT leads to a decomposition of approximation coefficients at

level j in four components: the approximation at level j+1, and the details in three

orientations (horizontal, vertical, and diagonal).

We have used wpdec2 function which is a two-dimensional wavelet packet

analysis function.

T = wpdec2(X,N,'wname') returns a wavelet packet tree T corresponding to the

wavelet packet decomposition of the matrix X, at level N, with a particular wavelet.

wname is the name of wavelet filter used. We have used db4 filter from

Daubechies wavelet filter family. We have found empirically that db4 is optimum filter

for our application in terms of efficiency of retrieval and time complexity.


35


36/58

Algorithm : Feature Matching using wavelet packet decomposition

1. Decompose the image using 2-D wavelet packet decomposition

2. Calculate energy corresponding to the terminal nodes of the tree T using

wenergy(T) function.3. Select energy of 5th to 16th terminal nodes of tree .

4. Calculate Euclidian distance for these nodes.

(4.3)

Where,

n denotes the no. of images in database.

wpd_query is the W.P.D. energy vector of query image,

wpd_database is W.P.D. energy vector of the database image.

wpd_dist is error difference or distance metric

W.P.D. =Wavelet Packet Decomposition




36


37/58

4.2.2.1 RESULTS (wavelet packet decomposition)


QUERY IMAGE:-


RETRIEVED IMAGES :-



37


38/58

QUERY IMAGE:-


RETRIEVED IMAGES :-



38


39/58

DISCUSSION: Co-occurrence matrix based image retrieval yields good results for images

containing certain structure.

Wavelet packet decomposition based image retrieval yields even more intended

results for images having specific structure.

This was the drawback of colour based image retrieval which is removed by

wavelet decomposition.


39


40/58

4.3 SHAPE

Shape may be defined as the characteristic surface configuration of an object, an

outline or contour. It permits an object to be distinguished from its surroundings by its

outline[12]. We have considered edge detection technique for extracting Shape

representation of an image.

we are using prewitt filter to determine the horizontal and vertical edges of the

query image [1].

Boundary-based shape representation only uses the outer boundary of the shape.

This is done by describing the considered region using its external characteristics i.e. the

pixels along the object boundary.

Changes or discontinuities in an image amplitude attribute such as luminance

value are fundamentally important primitive characteristics of an image because they

often provide an indication of the physical extent of objects within the image. Local

discontinuities in image luminance from one level to another are called luminance edges

[1].

(4.4)

(4.5)

This mask is used for Prewitt filter which we have used for calculating the

boundary feature of the image.


40


41/58

We can calculate the horizontal and vertical edges separately.Then we can

calculate the combined edges from these two edges. It is given as [1]

Combined edges = (4.6)

Where,

Icx=filtered result of image data with mask Px,

Icy=filtered result of image data with mask Py.

Here the order used is 2.5 , as we go on increasing the order the number of edges

become more as it will represent the small change in the pixel to pixel value. So the edges

will be more clear but for the analysis of image retrieval that much accuracy is not

necessary. So the order between 2 to 4 is sufficient.


41


42/58

Algorithm for calculating shape feature [1]:

Figure 4.12: Flowchart for calculating shape feature


42

Input the Image

Filter the image using Prewitt filter

for obtaining verticaledges.

Use complement of Prewitt filter for

obtaining horizontaledges.

Calculate the combined edges of

query image

Use this as standard shape feature

Calculate the standard deviation of

combined edges of query image


43/58

4.3.1 RESULTS (shape feature : edge detection)


QUERY IMAGE:


RETRIEVED IMAGES :


43


44/58


DISCUSSION: We have implemented boundary scanning technique (both horizontal and vertical)

for extracting shape features of an image.

Boundary scanning technique yields intended results for images having objects

with specific shape.

Hence it overcomes the drawback of colour and texture based retrieval.


44


45/58

4.4 INTEGRATION OF ALL FEATURE VECTORS

Each image feature such as colour, texture and shape is suitable for retrieval for a

particular class of images. Hence to implement efficient CBIR system for general

purposes, we have gone for integration of all four feature vectors viz. colour, co-

occurrence matrix, wavelet packet decomposition, edge detection.

We have each particular feature vector of following length:

Colour Feature Vector = 20;

Co-occurrence matrix Feature Vector =25;

Wavelet Packet Decomposition Feature Vector =12;

Edge Detection Feature Vector =25.

After INTEGRATION ,length of new vector will be:


45


46/58

Integrated Feature Vector = 82.

4.4.1 ASSIGNING DIFFERENT WEIGHT TO EACH INDIVIDUAL FEATURE.

We have given different weight to each individual feature depending on users

requirement. We multiply each individual feature vector by user specified weight. e.g. to

give 35% weightage to colour feature we multiply it with 0.35 . In similar way, we give

different weightage to remaining feature as per users requirement.

Figure 4.14: GUI designed for assigning weights

The importance of specific feature is increased by assigning more weight and

similarly decreased by assigning lesser weight.

If we assign 35% weight to colour feature, it means that colour feature is utilized

upto 35% of its total strength. In this context if we assign 10% weight to co-occurrence

matrix feature, then certainly colour feature gets more importance than co-occurrence

matrix feature.

Optimum weights are to be found empirically based on particular application.

4.4.2 G.U.I

The Graphical User Interface was constructed usingMatLab GUIDEorGraphicalUser Interface Design Environment. Using the layout tools provided by GUIDE, we

designed the following graphical user interface figure (gr_17_cbir.fig) for our CBIR

application:


46


47/58

Figure 4.15: Developed GUI in MATLAB GUIDE

4.2.4.1 Query Image selection:

Query Image can be selected by clicking Select pushbutton. Any image from computer

or external device connected to the computer can be selected as a query image.


47


48/58

Figure 4.16:Selection of query-image

4.2.4.2 Adjusting feature weights:

Feature weights can be adjusted with the help of slider movement.

Figure 4.17:Adjuting appropriate feature weights

4.2.4.3 Retrieval of images :

Images similar to the queryimage can be retrived byclicking Searchpushbutton.Top sixresults are displayed.


48


49/58

Figure 4.18:Image retrieval by clicking on search pushbutton.

4.5 Relevance feedback:

Specification of the query is very critical in CBIR. In fact, if the sample image is

located near to the boundary of the relevant cluster in the feature space, the first retrieval

may contain few relevant images. In this case, the query refinementmechanism is usefulto move the query towards the middle of the cluster of relevant images[19].We use the

sets of relevant and non-relevant images ( and ; ) specified by the user to

calculate a new query by applying the Rocchios formula [19]

(4.7)

Where Q is the previous query, , are the number of images in ,;and , are the feature vectors associated to the relevant and irrelevant images,

respectively. In (4.7), , , and parameters [with ] control the relative

importance of the previous query, the relevant images and the nonrelevant images,

respectively[19].


49


50/58

One of the results we have got is as follows

Figure 4.19:One of the results

Relevance feedback can be given by ticking the checkbox namedfeedback.

User is asked about the most similar image he /she thinks through command window

when he/she click on Search pushbutton.User has to the rank of the most similar image.

Figure 4.20: Users feedback through command prompt

After query refinement we got following results.


50


51/58

Figure 4.21: Results after giving relevance feedback

4.6 DATABASE

The image database that we used in our project contains 350JPGimages.

Few images are photographs of S.T.E.S. campus taken from digital camera.

Few images are photographs of AutoExpo.in Delhi taken from digital camera.

Remaining images have been randomly selected from the World Wide Web.

4.7 ACCOMPLISHMENTS

Demo 1.


51


52/58

Figure 4.21: Demo of retrieval(1)

Demo 2.


Demo 3.

Here query image is an image drawn in Paint software


52


53/58


Figure 4.24: Demo of retrieval(3)-improved results by assigning diff.weights.

Demo 4.

Here query-image is not in the database & is drawn with the help of Paint software.


53


54/58


Results are improved by giving feedback about 5th image as relevant & 4th as irrelevant.

Figure 4.26: Improved results( by relevance feedback )

CONCLUSIONS

Based on implementation of CBIR system on MATLAB we came to following

conclusion:


54


55/58

A. COLOUR:

Colour is a low level (pixel level) feature. It gives intended results for images

containing natural beauty and similar kind of images but it doesnt produce

intended search results for images having specific shape or structure.

Hence for building effective as well as efficient CBIR system, one should think

beyond colour based image retrieval.

B. TEXTURE:

Co-occurrence matrix based image retrieval yields good results for images

containing certain structure.

Wavelet packet decomposition based image retrieval yields even more intended

results for images having specific structure.

This was the drawback of colour based image retrieval which is removed by

wavelet decomposition.

C. SHAPE:

Boundary scanning technique yields intended results for images having objects

with specific shape.

Hence it overcomes the drawback of colour and texture based retrieval.

D. Each image feature such as colour, texture and shape is suitable for retrieval for a

particular class of images. Hence not any one of the feature can be used as

standalone feature for retrieval of images even though it produce excellent results

for a particular class of images.


55


56/58

E. For building effective as well as efficient CBIR system we have gone for

integration of all these features like colour, texture and shape.

F. Different weights are applied to each feature as per its importance depending on

user.

G. Optimum weights for respective features are decided empirically. We have found

optimum weights as:Colour:35%; Co-occurrence matrix:10%; Wavelet

Feature:20% and Shape:35% as far as general applications are considered .

H. Depending on the user and nature of database optimum values of weights for

respective image features can be different e.g. if database contains images of

historical importance having artistic drawings and constructions, then greater

weight should be assigned to shape and texture rather than colour.

I. Query refinement by Rocchios formula helps in moving query towards the middle

of the cluster of relevant images. Hence relevance feedback using query

refinement technique improves results.

FUTURE ASPECT

CBIR is still a developing science. As image compression, digital image

processing, and image feature extraction techniques become more developed,

CBIR maintains a steady pace of development in the research field. Current

techniques are based on low level features and there is a huge semantic gap

existing. Much more research work is needed for coming out with a reliable and

semantically competent system Furthermore, the development of powerful

processing power and faster and cheaper memories contribute heavily to CBIR

development. This development promises an immense range of future applications

using CBIR.


56


57/58

REFERENCES

1. K. Jalaja, Chakravarthy Bhagvati, B. L. Deekshatulu, Arun K. Pujari Texture

Element Feature Characterizations for CBIR:-Geoscience and Remote SensingSymposium,2005. IGARSS 05. Proceedings 2005 IEEEInternational Volume 2,

Issue , 25-29 July 2005 . IEEECNF

2. Rudinac.S.; Zajic.G.; Uscumlic.M.; Rudinac.M.; Reljin.B.:- Comparison of CBIR

Systems with Different Number of Feature Vector Components ;Semantic Media

Adaptation and Personalization, Second International Workshop on 17-18 Dec.

2007.IEEECNF

3. Barbeau Jerome, Vignes-Lebbe Regine, and Stamon Georges, A Signature based

on Delaunay Graph and Co-occurrence Matrix, Laboratoire InformatiqueA et

Systematique, Universiyt of Paris, Paris, France, July 2002.

4. K.P.Soman,K.I.Ramchandran ;Insight into Wavelets-From Theory to Practice ;

PHI publication,2003 edition.

5. Linda G. Shapiro, and George C. Stockman, Computer Vision, Prentice Hall,

2001.

6. R. Jain, R. Kasturi, and B. G. Schunck, Machine Vision, McGraw Hill

International Editions, 1995.

7. FOLDOC,Free On-Line Dictionary Of Computing, texture, May 1995, [Online

Document]

8. Texture, class notes forComputerized Image Analysis MN2, Centre for Image

Analysis, Uppsala, Sweden, Winter 2002

9. FOLDOC,Free On-Line Dictionary Of Computing, wavelet, May 1995, [Online

Document]

10.Lexico Publishing Group, LLC, shape, [Online Document]11.Benjamin B. Kimia, Symmetry-Based Shape Representations, Laboratory for

Engineering Man/Machine Systems (LEMS), IBM, Watson Research Center,

October 1999.


57
http://www.ieeexplore.ieee.org/xpl/RecentCon.jsp?punumber=4414369http://www.ieeexplore.ieee.org/xpl/RecentCon.jsp?punumber=4414369http://www.ieeexplore.ieee.org/xpl/RecentCon.jsp?punumber=4414369http://www.ieeexplore.ieee.org/xpl/RecentCon.jsp?punumber=4414369


58/58

12.FOLDOC, Free On-Line Dictionary Of Computing, co-occurrence

matrix, May 1995, [Online Document]

13. Stephen J. Chapman; MATLAB Programming for Engineers: second edition,

Bookman companion series.

14. HuiA Yu1, Mingjing Li2, Hong-Jiang Zhang2, Jufu Feng1;Color texture

moments for content-based image retrieval* Center for Information Science,

Peking University, Beijing 100871, China ,Microsoft Research Asia,49 Zhichun

Road, Beijing 100080 , China

15.Gonzalez & Woods,Digital Image Processing using MATLAB Pearson

Education, Third Edition,2009 reprint.

16.Remco C. Veltkamp, Mirela Tanas ; Content-Based Image Retrieval Systems: A

Survey; Department of Computing Science, Utrecht University

17. John Eakins,Margaret Graham; Content Based Image Retrieval; University of

Northumbria at Newcastle ;Report: 39-JISC Technology Applications Program

18. Shengjiu Wang, A Robust CBIR Approach Using Local Color Histograms,

Department of Computer Science, University of Alberta, Edmonton, Alberta,

Canada, Tech. Rep. TR 01-13, October 2001.

19. Anelia Grigorova, Francesco G. B. De Natale, Charlie Dagli, and

Thomas S. Huang; Content Based Image Retrieval by FeatureAdaptation and Relevance Feedback.IEEE Transactions on multimedia,

vol. 9, no. 6, October 2007

PUBLICATIONS:

T.D.Joshi,V.K.Kolhe,A.B.Kore;Implementation of common CBIR techniques,

A National conference on pervasive computing, 9-10 April 2010,Dept. ofElectronics & Telecommunication, Sinhgad college of Engineering, Pune-41,Maharahtra,India.

project report new 03

Documents