an approach to linear spatial filtering method based on anytime algorithm for real-time image...

7/29/2019 An Approach to Linear Spatial Filtering Method based on Anytime Algorithm for Real-time Image Processing

1/7

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 12, DECEMBER 2012, ISSN (Online) 2151-9617https://sites.google.com/site/journalofcomputing

WWW.JOURNALOFCOMPUTING.ORG 26

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

An Approach to Linear Spatial Filtering Method based

on Anytime Algorithm for Real-time Image ProcessingWyne Wyne Kywe and Kazuhito Murakami

AbstractReal-time image processing system requires not only correct but also for the imperfect timely output with deadline satisfaction. It still has the problem that to

realize the imperfect but usable result at available processing time. In order to solve the above problem, this paper proposes an approach to image enhancement method inspatial domain based on convolution and the concept of anytime algorithm for real-time image processing system. First, we construct sub-masks by dividing the filter

mask. Then, we evaluate anytime spatial function according to the concept of anytime algorithm for the outputs of linear spatial filtering. In order to produce the interme-

diate results, some of image enhancement tasks such as noise reduction, edge detection and sharpening are performed by these sub-masks step by step. The combinationof above image enhancement tasks can also be performed by giving the sub-mask number as parameters under time restriction. The experimental results show that the

intermediate result of each task and the overall result of combination of above tasks can be obtained at available processing time with better image details. It shows the

possibility of our proposed method and it is useful for the real-time image processing system under time restriction.

Index TermsAnytime algorithm, image enhancement, real-time image processing, spatial filtering.

1 INTRODUCTION

N real-time image/video processing system, if we seriously

consider the less processing time in a particular task, the

processing quality would becomes sacrifice and conversely, if we

adhere to processing quality too much, the processing time might

needed much time. The purpose of real-time image processing

system involves with the improvement of the quality of video

(image sequence) by enhancing the image in pre-processing like

noise reduction. Furthermore, many imaging applications are time

critical and are computationally intensive. For example, in image

transmission system, digital images require huge amounts of

space for storage and large bandwidths before transmission of

image, so it is necessary to process these images in pre-processing

such as filtering and enhancement. The outputs are required not

only the perfect but also the timely imperfect results with deadline

satisfaction. Moreover, the quality of image processing is usually

evaluated by high extraction rate or low error rate. It is easy and

clear to evaluate single image processing task, but if the system is

composed of many tasks, it would becomes difficult to evaluate

the combined processing result because of the results of image

processing vary according to the combination of the methods and

tasks. For example, if the system is composed ofNtasks and it is

needed tn processing time, when the time is restricted into t, here, t< tn, it is difficult to produce the intermediate result at available

processing time t. Forinstance, in image/video tracking or image

transmission which is necessary to realize the intermediate result

i.e., to achieve the better resolution with data transmissions and

computations as low as possible, at intermediate processing time t.

There are many methods for image/video processing results re-

ported in the real-time image processing from the different view-

points such as hardware platform such as FPGAs, DSP (Digital

Signal Processors), GPU, Hybrid and PC based systems, software

platform such as pipeline, parallel processing and algorithm. Al-

though there are many spatial filtering methods for real-time ap-

plication are discussed in [1], the approach to the spatial filtering

methods to solve time vs quality trade-off problem in real-time

image processing is not reported yet. So, in this paper, we intro-

duce an approach to linear spatial filtering method by the concept

of anytime algorithm in order to perform the task as sub-task in

pre-processing of real-time image processing under time restric-

tion from the viewpoint of algorithm. By dividing the filter mask

into small sub-masks, the intermediate result would be provided

by performing the tasks by these sub-masks. Like this idea, in real

time image processing system, a task can be divided into small

sub-tasks in order to perform the combination of tasks with some

restrictions like processing time and memory usage by giving the

partial overall result.

We already proposed how to apply the concept of anytime al-

gorithm to some of spatial filtering method like averaging fornoise reduction and Gradientby Prewitt filter for edge detection

[2], [3]. In this paper, we extend our method to linear spatial fil-

tering in order to provide the better solution of above time vs

qualtiy trade-off problem from the viewpoint of image quality

and/or processing time according to the features of anytime algo-

rithm [4], [5], [6], [7], [8].

Wyne Wyne Kywe is with the Graduate School of Information Science andTechnoloy at Aichi Prefectural University, Nagakute-shi, Aichi, 480-1198,

Japan.

Kazuhito Murakami is with the Department of Graduate School of Informa-tion Science and Technoloy at Aichi Prefectural University, Nagakute-shi,

Aichi, 480-1198, Japan.

I


2/7




This paper is organized into 4 sections. Section 2 ex-

plains about the proposed spatial filtering method comparedwith the conventional method, and anytime algorithm. Sec-

tion 3 provides the discussion about the experimental results

of proposed method. Finally, conclusion and the futureworks are described in section 4.

2SPATIAL FILTERING AND ANYTIME ALGORITHM

The fundamental process in digital image processing is the

image enhancement which is applied in every field for e.g.,medical image analysis, and analysis of images from satel-lites like weather map where the images are required to be

understood and analyzed. Moreover, there are large amount

of data usage to process an image in pre-processing steps

rather than that of intermediate and high level processing indigital image processing. Spatial filtering method is widelyused in image processing, either as a preprocessing step or

as a mean for gathering some interesting features such asedge, boundary, noise in an image for other image

processing processes like object detection, and video track-ing in image analysis.

Anytime algorithm is an algorithm which is differentfrom the traditional algorithm and it is satisfied the featuresthat the quality of result can be improved when the

processing time increases and the result can be produced at

anytime [4], [5], [6], [7], [8]. We applied this concept in im-plementing anytime function in order to perform the sub-

tasks and output the partial result in intermediate processingtime for the linear spatial filtering such as low pass filtering

and high pass filtering.

Conventional linear spatial filtering

In the conventional spatial filtering, there are two types:linear and non-linear. Linear filters such as mean and Gaus-sian for smoothing and Gradient operators such as Sobel,

Prewittand Canny filters for edge detection and basic hi-pass spatialfilter andLaplacian filter for sharpening.

In the conventional spatial filtering, generally, the re-

sponseR of an m x n mask at any point (x, y) in an image,the linear spatial filtering is considered by the followingexpression:

R = w1f1+ w2f2++ wmnfmn= =1 (1)

where the ws are the coefficients of mask, the fs are the

values of the image gray levels corresponding to the maskcoefficients, and mn is the total number of coefficients in the

mask. In general, linear filtering of an image fof size MxN

with a filter mask of size mxn is given by the expression:

a

ai

b

bj

jyixfjiwyxfyxwyxg ),(),(),(),(),(

(2)

Where

f= input image

g= output imagem = 2a+1

n = 2b+1

w = the elements ofmxn maskand, a and b are non-negative integers

Proposed linear spatial filtering

Fig.1. 3x3 filter mask

For 3 x 3 filter mask as shown in Fig-1, the response R at

any point (x,y) in the image is given by

R = w1f1+ w2f2++ w9f9= 9=1 (3)

In this spatial filtering method, 3x3 filter mask is dividedinto 8 sub-masks as shown in Fig.2 by increasing the place

of corresponding elements in each sub-mask. In this ap-proach, we construct many different patterns (i.e., 40320=8!)of divided sub-masks like HLAC features. Among 40320

patterns, 20738 different results with different patterns areobtained after removing the patterns with same results when

the image or filter mask is rotated to 180 degree. We will

report about the different patterns of linear spatial filtering inour later papers.

Fig.2. Divided sub-masks of 3x3 filter mask

Figure 2 is one of the patterns with the better result

among many other different patterns according to the expe-

riments. As shown in this figure, sub-mask 1 has two ele-ments and the first element is started from center as refer-

ence pixel in HLAC features and the other element is lower

element. In sub-mask 2, it has three elements defined by theelements of sub-mask 1 and added by the upper element.

Sub-mask 3 is defined by the elements of sub-mask 2, and

added by right element, and then upper left, upper right andso on in next sub-mask up to sub-mask 8.

Then, the operation is performed by divided sub-masksusing 8 steps. So, the response R at any point (x, y) in the

image can be performed by the following 8 steps:


3/7




Step-1 :R1 = w5f5+ w8f8

Step-2 :R2 =R1 + w2f2Step-3 :R3 =R2 + w6f6

Step-4 :R4 =R3 + w4f4

Step-5 :R5 =R4 + w1f1Step-6 :R6 =R5 + w3f3

Step-7 :R7 =R6 + w7f7

Step-8 :R8 =R7 + w9f9

In each step, the previous result is used to perform the

calculation of current steps in order to perform the task effi-ciently according to the properties of anytime algorithm.

Thus, 8 different intermediate responsesRs can be obtainedat each step with related processing time.

We applied this proposed method to smoothing process

by Gaussian filter, and sharpening process by basic hi-pass

spatialfilter using above patterns and edge detection processby Sobelfilter using another pattern which will be explained

in the later part.

(1) Smoothing by Gaussian filterIn anytime algorithmic linear spatial filtering, smoothing

process by using Gaussian 3x3 mask is done by applying

divided sub-masks as shown in Fig. 3. Figure 3 shows howto perform anytime algorithmic smoothing method by usingGaussian 3x3 mask. In particular, we applied the following

Gaussian 3x3 filter mask as an example.

(a)

(b)

Fig.3. (a) Gaussian 3x3 mask (b) Divided sub-masks ofGaussian

In general, the output image is evaluated by anytime lowpass spatial function which is modified from (2) for anytime

algorithmic low pass filtering and is given by

)4(]),(),([

),(

),(+),(=),( 111

1

11 jigyxf

yxw

yxwjigjig kkk

l

l

kkk

Where

k= 1, 2, , 8

i = 1, 2, ..., W1

j = 1, 2, ..., H1fk(x, y) = input imagegk(i, j) = current output

gk-1(i, j) = previous output

WandH= images width and height

wk(x, y) = the elements of 3x3 mask, here, wk are

positive for low pass filtering

(2) Edge detection by Sobel filterIn anytime algorithmic linear spatial filtering, edgedetection process by using Sobel 3x3 mask is done by

divided sub-masks as shown in Fig. 4. Figure 4 showshow to perform anytime algorithmic edge detectionmethod by using Sobel 3x3 mask.

(a)

(b)

Fig.4. (a) Sobel3x3 mask (b) Divided sub-masks ofSobel

In general, the output image is evaluated by anytime spa-

tial function forGradient methodand is given by


4/7




= 1 + , (5) = 1 + , (6) = + (7)

Where, = input imageG = output image

GX= gradient by horizontalGY= gradient by vertical

T= transformation on ,

(3) Sharpening by basic hi-pass spatial filterIn anytime algorithmic linear spatial filtering, sharpen-ing process by using basic hi-pass spatial, 3x3 mask isdone by divided sub-masks as shown in Fig. 5.

(a)

Fig.5. (a) Basic hi-pass spatial3x3 mask (b) Divided sub-masks ofbasichi-pass spatial filter

The output image is evaluated by anytime high pass spa-tial function for anytime algorithmic high pass filtering and

is given by

)8(),(),(++),(=),( 01 yxfyxcwgjigjig kkkk

Wherek= 1, 2, , 8i = 1, 2, ..., W1j = 1, 2, ..., H1

fk(x, y) = input image

gk(i, j) = current outputgk-1(i, j) = previous output

g0 = cwcentre(x, y) fcentre(x, y), c = constantWandH= images width and height

wk(x, y) = the elements of 3x3 mask, here, wk are

negative except from centre element

wcentre(x, y) for high pass filtering

In general, the output sharpened image is obtained by

, = , + (,) (9)

Where

, = output image, = input image(,) = filtered image

3DISCUSSION

In this section, we discuss about the experimental results of

noise reduction (smoothing) by Gaussian filter, edge detec-

tion by Sobel filter and image sharpening by basic hi-pass

spatial filter performed by the divided sub-masks as de-scribed in section 2. Then, the experimental result of the

combination of above 3 tasks is presented for the better im-

age enhancement under time restriction.

Experimental settingInput- Tasks: smoothing, edge detection, and sharpening- Parameter: Required sub-tasks number of each task

for e.g., sub-task 3 for smoothing, sub-task 4 for edge

detection and sub-task 8 for sharpening

Output- To produce the intermediate result of the combination

of 3 tasks by given parameter of sub-tasks

Many standard images are used from the book ofDigi-

tal Image Processing, 3rd ed, by Gonzalez and Woods whichis obtained from the following link:

http://imageprocessingplace.com/DIP3E/dip3e_book_im

ages_downloads.htm

Fig.6. Some of tested standard images with size 1024x1024

Experimental results

(a) Smoothing by Gaussian filterWe have done experiments for the smoothing (noise reduc-

tion) by using the standard images with size 1024x1024.
http://imageprocessingplace.com/DIP3E/dip3e_book_images_downloads.htmhttp://imageprocessingplace.com/DIP3E/dip3e_book_images_downloads.htmhttp://imageprocessingplace.com/DIP3E/dip3e_book_images_downloads.htmhttp://imageprocessingplace.com/DIP3E/dip3e_book_images_downloads.htmhttp://imageprocessingplace.com/DIP3E/dip3e_book_images_downloads.htm


5/7




First, we put the random noise to the standard input images,

then the noise reduction task is performed by the relatedsub-masks using Gaussian filter step by step. Figure-7 (a) is

the input standard Lena image and Fig. 7 (b) to (i) are the

experimental results of smoothed image performed by di-vided sub-masks. Here, we would like to show that the dif-

ferent intermediate results can be obtained by applying the

sub-masks of Gaussian filter even though the noise reduc-tion quality is evaluated by SNR.

(a)

Fig.7. (a) Standard Lena image with size 1024x1024 (b) to (i) smoothing

result images by divided 8 sub-masks using Gaussian filter

Figure 8 displays the performance curve of step vs quali-

ty of result i.e., the probability of number of reduced noise.It expresses that the average performance curve of the quali-

ty of noise reduction result gradually improves as the

processing time increases with related steps.

Fig.8. Performance curve of noise reduction by Gaussian filter

(b) Edge detection results by Sobel filterFigure 9 shows the experimental result of anytime algorith-

mic edge detection by divided sub-masks ofSobel3x3 filter.

We can see that 6 different results are obtained with requiredprocessing time and related steps. We can easily know the

fact that the quality of edge detection result at each step is

increasingly improved as the processing time increased asshown in Fig. 10.

(a)

Fig.9. (a) Original image with size 2048x2048 (b) to (g) edge detection

results by divided sub-masks ofSobelfilter

Fig.10. Performance curve of anytime algorithmic edge detection by Sobelfilter

(c) Sharpening results by basic hi-pass spatialfilterFigure 11 shows the experimental result of sharpening by di-

vided sub-masks ofbasic hi-pass 3x3 filter. We can see that 8

different results are obtained with required processing time and

related steps. We can know the fact that the quality of shar-

pened result at each step is increasingly improved as the

processing time increased as shown in Fig. 12.


6/7




(a)

Fig.11. (a) Original image with size 1024x1024 (b) to (i) sharpening resultimages by divided sub-masks ofbasic hi-pass filter

Fig.12. Performance curve of sharpening by basic hi-pass filter

Combination of image enhancement tasks

In this part, we express how to apply this proposed methodto the multiple steps spatial enhancement by combining

noise reduction, edge detection and sharpening tasks. So,

this section provides the experimental result of the combina-tion of the above 3 tasks for the image enhancement under

time restriction by using many standard images. Figure 13(a)shows that original mandrill gray image and (b) is the resultimage after smoothing by Gaussian sub-mask 3, (c) the re-

sult image of edge detection by Sobelsub-mask 4, and (d) isthe result image of sharpening by basic hi-pass spatial sub-

mask 8. As shown in this figure we can know the fact that

the intermediate overall result could be obtained by input-ting the sub-mask numbers as parameter.

This shows the possibility of the proposed method in or-

der to perform the combination of tasks in pre-processingsteps in real-time image processing with mega data usage

under time restriction.In the conventional method, the result or output of com-

bination of three tasks i.e., f1,f2, andf3 can be performed by

, = [1,,2,,3, ] (10)

In this proposed method, the result or output of the com-bination of three tasks can be performed by

, = [1,,2

,,3,] (11)

Where

,= result or outputT= an operator onfdefined over some neighborhood of

(x,y)

1, ,2,,and3, = input tasks 1, 2, and 3

= sub-task number of taskf1 = sub-task number of taskf2 = sub-task number of taskf3

In general, the result or output of the combination ofN

tasks can be performed by

, = [1, ,2

,, ,, ] (12)

(a) (b) (c) (d)

Fig.13. (a) Original mandrill gray image with size 1024x1024 (b) the result

image of smoothing by Gaussian sub-mask 3 (c) the result image of edge

detection by Sobelsub-mask 4 (d) the result image of sharpening by basic

hi-pass spatial sub-mask 8

Effectiveness

By dividing the mask into sub-masks, the following facts arerealized.- the different intermediate results can choose with the

related processing time

- different performance curves can be achieved- it is suitable for the real-time image processing system

that uses mega data and restricted processing time such

as image and video tracking, and image transmission

systems and for the scheduling of tasks under time con-straint etc.

- it is one of the idea to solve the time vs quality trade-offproblems encountered in real-time system by dividingthe task into small sub-tasks


7/7



4CONCLUSION AND FUTURE WORKS

We proposed an approach to linear spatial filtering methodby dividing the mask into sub-mask using the concept of

anytime algorithm for the real-time image processing systemunder time constraint. It is applied to the smoothing by

Gaussian filter, edge detection by Sobel filter and sharpen-ing by basic hi-pass filter as an example. Then, it is applied

to the multiple steps spatial enhancement for the combina-tion of noise reduction, edge detection and sharpening tasks.We constructed anytime spatial functions such as low-pass

spatial filtering, gradient method, and high-pass spatial fil-

tering according to the concept of anytime algorithm. The

output images are evaluated by using these functions. Theexperimental result of the combination of the above threetasks shown that the intermediate result can be obtained at

intermediate processing time. So, it is useful for the imageenhancement as a pre-processing step in real-time image

processing system under time constraint. It is also useful forthe implementation of embedded systems application.

In conclusion, this proposed method could be applied forthe image enhancement such as low-pass filtering, high-pass

filtering in spatial domain for the linear filtering under time

restriction. It can be applied not for the spatial filtering me-thod only, but also for the other image processing method

i.e., course-to-fine, and conditional methods. For the futureworks, we will extend this idea to the morphological

processing such as opening, closing, erosion and dilation,and construct the anytime algorithmic image processing li-

brary.

REFERENCES

[1] R.K. Sandhu, P.S. Maan, A spatial-domain filter for digital image

De-noising used for Real time applications,International Journal of

Computer Science and Technology, IJCST Vol. 2, Issue 3, September

2011.

[2] W.W. Kywe, D. Fujiwara and K. Murakami, Scheduling of Image

Processing Using Anytime Algorithm for Real-time System,Proc. ofthe 18th International Conference on Pattern Recognition

(ICPR2006), Vol.3, pp.1095-1098, Hong Kong/ China, 2006/08.

[3] W.W. Kywe and K. Murakami, Anytime Noise Reduction and Edge

Detection Algorithms for Time-Restricted Image Processing System,

Proc. of the 15th Japan-Korea Joint Workshop on Frontiers of Com-

puter Vision, pp.65-70, Andong/South Korea, 2009/02.

[4] T. Dean and M. Boddy, An Analysis of Time-Dependent Planning,

Proc. AAAI-88, pp.49-54, AAAI, (1988).

[5] S. Zilberstein, Using Anytime Algorithms in Intelligent Systems,

AI Magazine, vol. 17, no. 3, pp. 73-83, (1996).

[6] S. Zilberstein and S. J. Russell. In S. Natarajan (Ed.), Approximate

Reasoning Using Anytime Algorithms,Imprecise and Approximate

Computation, Kluwer Academic Publishers, (1995).

[7] J.Grass and S. Zilberstein. In M. Pittarelli (Ed.), Anytime Algorithm

Development Tools,SIGART Bulletin Special Issue on Anytime Al-

gorithms and Deliberation Scheduling, 7(2):20-27, (1996).

[8] S. Zilberstein Ph.D dissertation, Operational Rationality through

Compilation of Anytime Algorithm, Computer Science Division,

University of California at Berkeley, (1993).

[9] R.C. Gonzalez and R.E. Woods, Digital image processing,Prentice

Hall, 2nd Edition, 2002.

[10] J.R. Parker, Algorithms forimage processing and computer vision,

John Wiley & Sons, Inc. U.S.A, 1997.

Wyne Wyne Kywe received the B.Sc. (Honors) degree in Mathematicsfrom Yangon University, Myanmar in 1994 and M.S degree in InformationScience from University of Computer Studies, Yangon, Myanmar in 1998.She was an assistant lecturer in University of Computer Studies, Yangon,

Myanmar from 1998 to 2005. She is a Ph.D. student at Graduate School ofInformation Science and Technology, Aichi Prefectural University, Japan.

She is a member of IEICE and ORSJ in Japan. Her research interests in-clude digital image processing, image analysis, computer vision, patternrecognition, anytime algorithm, operation research and linear programming.

Kazuhito Murakami received the B.S. degree in Physics from NagoyaUniversity, Japan in 1984 and Ph.D. degree in Engineering also fromNagoya University in 2002. During 1984-1991, he stayed in NagoyaMunicipal Industrial Research Institute, and during 1991-1998 inChukyo University, respectively. From 1998, he is in Aichi PrefecturalUniversity. He is a member of the Institute of Image Information andTelevision Engineers, Japan. He received The RoboCup Federation RSJAWARD in 2002, Best Student Paper Award in 2007, 2nd place in Ro-boCup 2009 Small Size League, etc. His research interests are Houghtransform,facei mage processing, industrial application system of imageprocessing, multi-camera system, digital signage, and RoboCup.

an approach to linear spatial filtering method based on anytime algorithm for real-time image...

Documents