06.10.digital image stabilizing algorithms based on bit-plane matching

8/14/2019 06.10.Digital Image Stabilizing Algorithms Based on Bit-plane Matching

1/6

KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 617

DIGITAL IMAGE STABILIZING ALGORITHMS BASED ON BIT-PLANE MATCHINGSung-Jea KO, Senior Member IEEE, Sung-Hee Lee and Kyung-Hoon LeeDepartment of Electronics Engineering, Korea University5-1 Anam-Dong, Sungbuk-Ku, Seoul 136-701, KoreaEmail: [email protected]

Abntract-In this paper, we present a new digi tal imag estabilization (DIS) schem e based on bit-plane matching(BPM). The proposed DIS system performs motion es-timation using 1-bit planes which are extracted from avideo sequence. This motion estimatio n technique can berealized using only Boolean functions which have signif-icantly reduced computational complexity, while the ac-curacy of motion estimation is maintained. In the sec-ond part of this paper, a median-based motion correctionscheme is proposed which is robust to various irregularconditions such as moving objects and intentional pan-ning. Simulation results show that the proposed DIS al-gorithm exhibits better performance compared with ex-isting other algorithms when applied to real video signal.

I. INTRODUCTIONImage stabilization is the process of generating a com-

pensated video sequence where image motion by thecameras undesirable shake or jiggle is removed [1]-[5].The recent DIS systems are realized using digital im-age processing techniques instead of mechanical motiondetection techniques using gyro sensors or fluid prism

The image stabilization task can be subdivided intotwo basic systems, namely: i) the motion estimation sys-tem and ii) the motion correction system. In general, themotion estimation system generates several local motionvectors from subimages in the different position of theframe using a block matching algorithm (BMA). Themotion correction system determines the global motionof a frame by appropriately processing these local mo-tion vectors, and decides whether the motion of a frameis caused by undesirable fluctuation of the camera or in-tentional panning. The stabilized image is generated byreading out the proper block of fluctuated image in theframe memory [3]-[5].

PI-[51.

The full-search(FS) BMA under the mean absolute dif-ference (MAD) and mean square error (MSE) criteriacan be considered as an optimal solution for motion es-timation [6],[7]. However, the FS BMA requires largeamount of computations which causes time delay, andrequires complex hardware architecture [7]-[9]. n thispaper, we present a new motion estimation techniquebased on the bit-plane matching (BPM) for the DIS sys-tem. The proposed algorithm performs binary motionestimation using 1-bit planes which are extracted froma video sequence. This motion estimation technique canbe realized using only Boolean functions which have sig-nificantly reduced computational complexity, while theaccuracy of motion estimation is maintained. Simulationresults show tha t t he performance of the BPM-based mo-tion estimation algorithm is comparable to tha t of the FSBMA.

Various algorithms have been developed to estimatethe global motion of a frame from local motion vectors[3]-[7],[11]. Most of these algorithms are complicated,and thus are not simple to implement. In this paper, wepropose a simple and robust decision algorithm for deter-mining the global motion vector. In the proposed algo-rithm, the global motion vector of a frame is determinedbased on the order statistics of current local motion vec-tors and past global motion vectors. It is shown tha t t heDIS system using the proposed motion estimation andcorrection algorithms is less sensitive to irregular condi-tions such as moving objects and intentional panning.

This paper is orgainzed as follows: The proposed mo-tion estimation and correction algorithms are presentedin Sections I1 and 111, respectively. Simulation resultsare given in Section 1V and concluding remarks are inSection V.

This work was supportedby SAM SUNG ElectronicsCo.

Manuscript received June 17, 1998 0098 3063/98 $10.00 1998 IEEE

uthorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.


2/6

618 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

Fig. 1. Example of generating bit-planes from a gray-sclae image, (a) an original image, (b) eight bit-plane images.

11. M O T I O NESTIMATIONSI NGBIT-PLANEMAGESBefore describing the proposed method t o estimate lo-

cal motion vectors, we introduce the bit-plane decompo-sition of a gray-scale image.A . Bi t - p l a n e D e c o m p o s i t i o n of a Graysca le Im age

Let the graylevel of the pixel at location (z,y) in thet-th image frame with 2K graylevels be represented as

where a k , O 5 h 5 K - 1, is either 0 or 1. Let the k-th order bit-plane image be denoted by b t k ( z , y ) . Thisplane contains all the k-th order (ak) its. For the caseof the 8-bit image, an image is composed of eight 1-bit planes b k ( z , y ) N b$(z,y), ranging from plane 0 toplane 7. Fig. 1shows eight bit-planes decomposed froma grayscale image. bi(z , ) contains all the least signif-icant (lowest order) bits comprising pixels in the imageand b$(a, y) contains all the most significant (highest-order) bits. Note that only the higher order bit-plane im-ages contain visually significant data whereas the otherbit-planes contribute to more subtle details within theimage.B . M o t i o n E s t i m a t i o n B a s ed o n t h e B i t - p l a n e M a t c h i n g

Local motion vectors are estimated from four subim-ages (SI, 2, Ss,5 4 ) placed in appropriate positions in

the bit-plane as shown in Fig. 2. Each motion vector of asubimage in the current bit-plane image is determined byevaluating bit-plane matching over subimages in the pre-vious bit-plane and selecting the subimage which yieldsthe closest matching. This approach assumes tha t allpixels within the subimage have uniform motion and therange of the motion vector is constrained by the searchwindow.

Let the size of each subimage be M x N and a searchwindow be ( M + 2 p ) x ( N +2q) . For bit-plane matching,we define the correlation measure given by

~ ( m ,)= C b k t ( z ,y) CBbkt - ' ( z + m, y +n) (2 )(",Y)ES.

where bkt(a, ) and bkt-'(z, y ) , respectively, are the cur-rent and previous k-th order bit-planes, and @ is theexclusive-OR operation.

At each ( m , n ) , p 5 m 5 p and -q 5 n 5 q , withinthe search range, the proposed matching method calcu-lates Ci(m,n)which is the number of unmatched bitsbetween the reference subimage in the current bit-planeand the compared subimage in the previous bit-plane.The smallest Ci(m, ) yields the best matching for Si ,and thus local motion vector V; from Si is obtained asvit = arg min{Ci(m, n), p 5 m 5 p , - q 5 n 5 q } . (3)

uthorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.


3/6

KO ,Lee and Lee: Digital Image S tabilizing Algorithms Based on Bit-Plane Matching 619

! preuioiis bit-plane mageI -_-.____--__I-___Fig. 2. Estimation of local motion vectors from four subima ges n a bit-plane.

This motion estimation technique can replace thearithmetic calculations of BMA's based on conventionalMAD and M SE criteria with simple Boolean exclusive-OR operations, and thus has significantly reduced com-putational complexity.

Since the proposed DIS system performs motion esti-mation using a single bit-plane, it is important to selectan appropriate bit-plane for bit-plane matching. In thispaper, the 4-th order bit-plane, b;(z,y), is utilized toestimate the local motion vector since it contains boththe global information and details of the original image.To show that b:(z, y) is suitable for motion estimationbased on BPM, an example for BP M using 1-D signalsis presented in Fig. 3.. Fig. 3(a) shows an 1-D signalwhich is one horizontal scan line of a real image with256 graylevels, and Fig. 3(b) is the same scan line fromthe previous frame. Using (2) , we computed the correla-tion measure between two binary signals from the scanlines at each bit-level. Fig. 3(c) illustrates the s imulationresults. It is seen that the correlation measure from the4-th bit-level exhibits the steepest gradient around theminimum point.

111. MOTION ORRECTION UNDER IRREGULARCONDITIONS

In this section, we introduce the motion correction sys-tem to cope with irregular conditions such as movingobjects and intentional panning that degrade the perfor-mance of the DIS system.

Fig. 4 shows the proposed motion correction systemwhich consists of the decision unit, the integration unit,

and the motion compensation unit. The motion correc-tion system determines the global motion of a frame byappropriately processing local motion vectors, and de-cides whether the motion of a frame is caused by un-desirable fluctuation of the camera or intenti6nal pan-ning. The stabilized image is generated by reading outthe proper block of fluctuated image in the frame mem-ory.

In an image with motion, some subimages with mov-ing objects can produce motion vectors which are signif-icantly different from the other motion vectors. Fig. 5shows an image which has moving objects in some subim-ages. Fig. 6(a) and (b ) show the correlation measurescalculated using (2 ) from subimage SI which has nomoving object and subimage S4 with moving objects,respectively. In Fig. 6, for display, the correlation mea-sures are normalized using l -* here C,,, is themaximum Cj(m,n) within the search range. It is seenthat there does not exist a distinct maximal correlationvalue in S4.

In general, motion vectors from the subimages withmoving objects are not reliable and should be excludedfrom the global motion decision process. Moreover, sincethe hand movement is relatively slow than the frame rateof the video camera, two successive frames fluctuated bycamera's shake should have similar global motion.

Based on these properties of camera's motion, we pro-pose a simple and robust motion correction scheme whereglobal motion decision is performed using current localmotion vectors (V:, V i, Vi, :) and the previous global

uthorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8 2009 at 04:15 from IEEE X lore. Restrictions a l .


4/6

620 IEEE Transactions on Consumer Electronics,Vol. 44, No. 3, AUGUST 1998

-8 .o -4 -2 0 2 4 0 Bhorizontal displacement

(c>Fig. 3. An example of B P M using 1- D signals, (a) a 1-D signal which s one horizontal scan line of a real image, (b ) the same scan linefrom th e previouse frame, ( c ) he correlation measure between two binary signals from th e scan lines at each bit-level. (+: bit-level

6, 0: bit-level 5, *: bit-level4, : bit-level 3, X: bit-level 2) .

Fig. 4. Basic structure of the motion correction system.

motion vector V,"-'.global motion vector is obtained by

In the proposed algorithm, the

Vi = median{Vi, Vi, V i ,V i ,v,"-'} (4)Here the median of vectors is determined by seperatelyselecting medians of each vector elements.

Local motion vectors affected by undesirable condi-tions such as moving objects can be viewed as impulses.

It is known that the median filter is very effective in elim-inating impulses. Therefore, the median-based methodin (4 ) can exclude such abrupt local motion vectors andproduce a global motion vector similar to the previousone.

After determining the global motion vector, the mo-tion correction system decides whether the mot ion of aframe is caused by camera's motion or intentional pan-ning. For this decision, the global motion vector of a

'


5/6

KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 62 1

Sequence number

Fig. 5. An image with moving objects.

RMSERPM I EPM I BPM

( 4 (b)Fig. 6. Correaltion measures, (a) from subimage S1 which has no motion, and (b) rom subimage Sb which contains moving objects.

234

frame is integrated with a damping coefficient, and theintegrated motion vector designates the final motion vec-tor of a frame for motion correction. The integrated mo-tion vector V, for estimating intent ional panning is givenby

vat= DIVat-l + V,t (5)where Q t is a global motion vector and D1(0 < D1 < 1)is a damping coefficient for smooth panning.

0.0373 0.4886 0.03330.0296 0.3000 0.02190.0518 0.3409 0.0362

IV. SIMULATIONESULTSTo evaluate the motion estimation performance of the

proposed BPM algorithm, we compare it with two exist-ing DIS algorithms, namly, representative point match-ing (RPM) [3] and edge pattern matching (EPM) [4].The performance is evaluated using the root mean squareerror (RMSE) based on the FS BMA under the MADcriterion. The RMSE is given by

where (zn,yn) is the motion vector from the FS BMA,and (&, ijn) is that from aformentioned algorithms. Fivereal image sequences with a resolution of 640 x 240 and

TABLE IRMSES ASSOCIATED WITH LOCAL MOTION YECTORS.

I I I1 I 0.0334 I 0.3285 1 0.0205

I5 1 0.0373 1 0.5002 1 0.0229

150 frames are utilized for simulation.The RMSEs associated with the local motion vector

are summarized in Table I. It is seen that BPM exhibitsbetter performance than RPM and EPM.

Table I1 summarizes the RMSEs associated withglobal motion vectors. It is interesting to observe tha tthe RMSEs of BPM are always smaller than those ofRPM and EPM.

These experimental results indicate tha t the proposedBPM-based DIS system exhibits good performance com-parable to the FS BMA.

'


6/6

622

Sequence number12

IEEE Transactions on Consumer Electronics,Vol. 44, No. 3, AUGUST 1998

RMSERPM EPM BPM0.0285 0.1223 0.01780.0308 0.2082 0.0260

TABLE I1RMSES ASSOCIATED WITH GLOBAL MOTION VECTORS.

34 0.0212 0.0585 0.01640.0349 0.0827 0.0242I I I5 1 0.0374 I 0.1690 1 0.0242

V. CONCLUSIONIn this paper, we proposed a BPM-based DIS system

which performs motion estimation using 1-bit planes.It was shown experimentally that the performance ofBPM is very close to th at of the FS BMA n terms ofRMSE. Moceover, B P M can be realized using only sim-ple Boolean functions, and thus is more suitable for VLSIimplementations than existing algorithms. It was alsoshown that the median-based motion correction schemeis robust to irregular conditions such as moving objects.Simulation results show that the proposed DIS algorithmis a computationally efficient a lternative to existing DISalgorithms.

REFERENCESM. Oshima, T. Hayashi, S. Fujioka and T. Inaji, VHS cam-corder with electronic image stabilizer, IEEE Trans. on Con-aumer Electronics, vol. 35, no. 4, pp. 749-758, Nov. 1989.K. Sato, S . Ishizuka, A. Nikami, and M. Sato , Cont rol tech-niques for optical image stabilizaingsystem , IEEE Trans. onConsumer Electronics, vol. 39, no. 3, pp . 461-466, Aug. 1993.K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Ki-tamura, Automatic image stabilizing system by full-digitalsignal processing, IEEE Trans. on Consumer Electronics,vol. 36 , no. 3, p. 510-519, Aug. 1990.3.-K. Paik, Y.-C. Park, a nd De-W.Kim, An adaptivem otiondecision system for dig ital image stabilizer based on edge pat-tern matching, IEEE Trans. on Consumer Eleclronics, vol.38, no. 3, pp. 607-615, Aug. 1992.T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T.Imaide, Electronic image stabilizer for video came ra use,IEEE Trans. o n Consumer Electronics, vol. 36 , no . 3,pp .520-525, Aug. 1990.H. G. Musmann, P. Pirsch, and H. J. Gralleer, Advances npicture coding, Proc. IEEE, vol. 73, no. 4, p. 523-548, Apr.1985.B. Liu and A. Zaccarin, New fast algorithm s for th e estima-tion of block motion vectors, IEEE Trans. on Circuits andSydems for Video Technology, vol. 3, no. 2 , pp. 148-157, Apr.1993.3 . Lu and M. L. Liou, A simple and efficient search algo-rithm for block-matching motion estimation, IEEE Trans.

on Circuits and Sys tems for Video Technology, vol. 7, no. 2 ,pp. 429-433, Apr. 1992.H. Gharavi and M. Mills, Blodunatc hingmot ion estimationalgorithms-new results, IEEE B a n d . o n Circuits and Sys-tems, vol. 37, no. 5, pp. 649-651, May 1990.[lo] Y. T. se and R. L . Baker, Global zoom/pan estimation andcompensatio n for video compression, Proceedings of ICASSP91 , Toronto, Ontario, Canada, pp. 2725-2728, May 1991.

[113 M. Okada, Video camera and video signal reproducing ap-paratus with shake detection and correlation operation, U.S.Patent, no. 5,502,484, Mar. 1996.

[9]

S u n g - J e a KO eceived the Ph.D . degree in1988 and the M.S. degree in 1986, both inElectrical and Computer Engineering, fromState University of New York at Buffalo,an d the B.S. degree in Electronic Engineer-irg at Korea University in 1980. In 1992,he joined the Depa rtment of Electronic En-gineering at Korea University where he iscurrently an Associate Professor and Grad-

uate Chairman. From 1988 to 1992, he was an Assistant Pro-fessor of Electrical and Computer Engineering at the Universityof Michigan-Dearborn. From 1981 to 1983, he was with DaewooTelecom Corporation where he was involved in research and de-velopment on dat a communication systems. He received the Hae-Dong best paper award from the Ins titu te of Electronics Engineersof Korea (1997), and the best paper award from the IEEE AsiaPacific Conference on Circu its an d Systems (1996). Dr. KO s cur-rently a Senior Member in the IEEE and a programs chair (1996- Prese nt) of th e IEEE Seo ul Sectio n of Korea Council. His cur-rent research interests are in t he areas of digital signal and imageprocessing, and multimedia communications.

Sung-Hee Lee received the B.S. degreein information engineering and M.S. degreein computer science form Korea Universityin 1993 an d 1995, respectively. He is nowa Ph.D. candidate in electronic engineer-ing with the Department of Electronic En-gineering at Korea University. In 1995, hejoined the Research Institute for Informa-tion and Communication Technology, where

he is currently a research engineer. His research interests are inthe areas of wavelets and nonlinear signal and image processing.

Ky u n g - Ho o n Lee received the B.S. degreeand M.S. degree in electronic engineeringform Korea University in 1992 and 1994,respectively. He is now a Ph.D. candidatein electronic engineering with the Depart-ment of Electronic Engineering at KoreaUniversity. In 1993, he joined the ResearchInstitute for Information and Communica-tio n Technology, where he is currently a re-

searcher. His research interests are i n th e areas of nonlinear signaland im age processing including morphological and order statisticsfiltering.

06.10.digital image stabilizing algorithms based on bit-plane matching

Documents