TEXTURE-BASED 3D IMAGE RETRIEVAL FOR MEDICAL APPLICATIONS

X. Gao, Y. Qian, M. Loomes, R. Comley, B. Barn , A. Chapman , J. Rix

Middlesex University, UK

R. Hui

Department of Neurosurgery, General Navy Hospital, P.R.China

MIRAGE(Middlesex medical Image Repository with a CBIR ArchivinG Environment)

Aim: To develop a repository of medical images benefiting MSc and research students in the immediate term and serve a wider community in the long term in providing a rich supply of medical images for data mining, to complement MU current online e-learning system.

http://image.mdx.ac.uk/

JSIC

Innovation in the use of ICT for education and research.

http://www.jisc.ac.uk/

Proposed Framework for MRIAGE

GIFT(GNU Image Finding Tool)

GIFT is open framework for content-based image retrieval and is developed by University of Geneva.

Query by example and multiple query

Relevance Feedback

Distributed architecture (Client - Server)

MRML---C-S communication protocol

Demo:

GIFT Framework

Framework of Content-Based Image Retrieval

Texture-based 3D Brain Image Retrieval

Current Content-Based Image Retrieval (CBIR)

Content-based image retrieval system

QBIC, Nectar, Viper, etc.

Visual feature extraction from 2D image

Content-based 3D Brain Image Retrieval

Shape-based

Proposed framework for 3D image retrieval

3D Texture Feature Extraction

1) 3D Grey Level Co-occurrence Matrices (3D GLCM)

2) 3D Wavelet Transform (3D WT)

3) 3D Gabor Transform (3D GT)

4) 3D Local Binary Pattern (3D LBP)

1) 3D Grey Level Co-occurrence Matrices (3D GLCM)

3D GLCM is two dimensional matrices of the joint probability of occurrence of a pair of gray values separated by a displacement d = (dx,dy,dz).

52 Displacement vectors:

4 distance * 13 direction = 52

4 Haralick texture features:

energy, entropy, contrast and homogeneity

Feature vector:

208 components (=4 (features) * 52 (matrices)).

2) 3D Wavelet Transform (3D WT)

3D WT provides a spatial and frequency representation of a volumetric image.

2 scales of 3D WT

Mean and Standard deviation

Feature vector:

30 components (2 (features) +15 (sub-bands))

3) 3D Gabor Transform (3D GT)

3D GT generates a set of 3D Gabor filters

Gabor filters

Gabor Transform:

144 Gabor filters

4 (F) *6(θ)*6(Φ) =144

Mean and Standard deviation

Feature vector:

288 components (2 (features) +144(filters))

zFyFxFjzyxgFzyxg cossinsincossin2exp,,,,,,,^

iiii FzyxgzyxfGT ,,,,,*,, 144...3,2,1i

4) 3D Local Binary Pattern (3D LBP)

Local binary pattern(LBP) is a set of binary code to define texture in a local neighbourhood. A histogram is then generated to calculate the occurrences of different binary patterns.

59 binary patterns

Feature vector:

177 components (=59(patterns)*3(planes)

Similarity Measurement

Histogram Intersection(3D LBP)

Normalized Euclidean distance (3D GLCM,3D WT,3D GT)

i

ii IQIQD ,min,

2

,

i i

ii IQIQD

Experiment Results

Processing and Query time

Methods Processing time Query time

3D GLCM 10.65s 0.83s

3D WT 2.03s 0.11s

3D GT 14.3m 0.31s

3D LBP 0.78s 0.29s

Conclusion and Future work

Four 3D texture methods are exploited and evaluated in 3D MR image retrieval.

Future work:

Test on the larger dataset

Find the best 3D texture representations

Feature dimension reduction

Combinations of some texture descriptors

Plug 3D image retrieval into GIFT framework.

Thank You.