1

Contourlet-Based Edge Extraction for Image Registration

Shirin Mahmoudi Barmas and Shohreh Kasaei Dept. of Computer Engineering, Islamic Azad University, Science and Research Branch, Tehran, Iran.

sh_mh1@yahoo.com Dept. of Computer Engineering, Sharif University of Technology, Tehran, Iran. skasaei@sharif.edu

Abstract Image registration is a crucial step in most image processing tasks for which the final result is achieved from a combination of various resources. In general, the majority of registration methods consist of the following four steps: feature extraction, feature matching, transform modeling, and finally image resampling. As the accuracy of a registration process is highly dependent to the feature extraction and matching methods, in this paper, we have proposed a new method for extracting salient edges from satellite images. Due to the efficiency of multiresolution data representation, we have considered four state-of-the-art multiresolution transforms –namely, wavelet, curvelet, complex wavelet and contourlet transform- in the feature extraction step of the proposed image registration method. Experimental results and performance comparison among these transformations showed the high performance of the contourlet transform in extracting efficient edges from satellite images. Obtaining salient, stable and distinguishable features increased the accuracy of the proposed registration process. Keywords: image registration, remote sensing, satellite images, edge extraction, dual-tree complex wavelet transform, curvelet transform, and contourlet transform.

1. Introduction Image registration is the process of overlaying images (two or more named reference and sensed images) captured from the same scene but at different times and view points, or even by using different sensors. Therefore, it is a crucial step of most image processing tasks in which the final information is obtained from a combination of various data sources. These include image fusion, changes detection, robotic vision, archeology, medical imaging, and multichannel image restoration [1]. Typically, image registration is required in remote sensing applications such as change detection, multispectral classification, environmental monitoring, image mosaicing, weather forecasting, super resolution images, and integrating information into geographic information systems (GIS) [2]. It is also used in biomedical image processing applications for combining computer tomography (CT) and magnetic resonance imaging (MRI) data to obtain more complete information about the patient, monitoring tumor growth, treatment verification, and comparison of patients’ data with anatomical atlases [3]. It is also used in cartography (for map updating), computer vision (for target localization), automatic quality control, motion analysis, and target tracking. It is also employed in 3-D domain (for depth reconstruction from stereo images), general 3-D reconstruction, and security purposes (for recognizing hidden weapons), to name a few. In general, the majority of automated registration methods consist of the following four steps: feature extraction, feature matching, transform modeling, and image resampling. In the feature

2

extraction step, manually or preferably automatically, salient and distinctive features are extracted [4, 5, 6, 7]. In the feature matching step, the correspondence between the features detected in the sensed image and those detected in the reference image is established. Various feature descriptors (such as B-Splines, chain code, snake-model and length code algorithms [8]) are reported in the literature. In order to match the extracted features, a similarity measures (such as iterative closest points (ICP), Bessel algorithm, geometric ICP, least squares error (LSE), normalized cross correlation (NCC), average magnitude difference function (AMDF), and maximization of mutual information (MMI)) are used. In the transform modeling step, the parameters of the mapping function are computed by means of the established feature correspondence. Finally, in the image resampling step, the sensed image is registered by means of the mapping function. In this paper, we have focused on the feature extraction step and have tried to extract the most dominant and stable features from panchromatic satellite images in a fully automatic manner. Due to the saliency of edge features in panchromatic images and also their stability against environmental and illumination changes, here we have extracted image edges as our primary features. We have then extracted edge corners as our required control points (CPs). Consequently, more accurately extraction of edges leads to better CP detection which in turn improves the registration results. Note that the accuracy of extracted edges and the required computational cost of the algorithm form the measures to evaluate different methods. In the proposed method, multiresolution transforms are considered to provide us with a powerful tool for increasing the accuracy of extracted edges while decreasing the computational cost (due to their dimension reduction property). As such, different multiresolution methods such as, real wavelet transform (WT), dual-tree complex wavelet transform (CWT), curvelet transform (CV), and contourlet transform (CT) are considered. As the curvelet transform was proposed in continues domain, it could not implement in the discrete domain easily, and there isn’t any implementation of it. So, we consider only the theorem concerns of it. Our conducted experiments on a variety of remotely sensed images showed that regarding the accuracy and obtaining long continuous edges in resulted edge maps, the contourlet transform outperforms the WT and CWT. We reached the registration accuracy of 0.14 per pixel using the contourlet transform. Moreover, considering the computational cost, the WT is faster than the dual-tree complex wavelet, curvelet and contourlet transforms. However, it results in isolated points (not smoothly connected lines). Also, the complexity of the CWT makes it time consuming (more than five times more than the WT and three times more than the CT). Using our proposed algorithm, the distance between the edges extracted from the original and registered images is about 0.013 per pixel which shows an amount of 0.17% improvement when compared to other available algorithms [9]. Also, contourlet extracted edges are more acceptable than the edges extracted using CWT. The reminder of this paper is organized as follows. In Section 2, the related works in the image registration area are reviewed. In section 3, a brief review on contourlet transform and its superiority against WT, CV and CWT is presented. In Section 4, our proposed method for extracting salient edges and corners from satellite images is described. In Section 5, the experimental results are presented, and finally Section 6 concludes the paper.

3

2. Related Works In general, the majority of registration methods consist of the following four steps: feature extraction, feature matching, transform modeling and image resampling. The related literature of each step is briefly explained next. 2.1 Feature extraction: To manually or preferably automatically extract salient and distinctive features such as, closed boundary regions [10], edges [4, 11, 12] and points [13, 14]. For further processes, these features can be represented by their point representatives (centers of gravity, line endings, distinctive points, moment or differential descriptors for curves [15]), which are called control points in the related literature. In the proposed method, we used contourlet transform to extract strong, smooth, and connected edges from input images. There are many edge extraction methods reported in the related literature. These could be grouped in two main categories: point-wise and region-wise. In point-wise methods, only isolated pixel values take part in the edge extraction process. They include highpass, and bandpass filtering as well as Robert, Sobel, Prewitt, and Canny edge detectors. These methods have low computational costs but they cause ringing effects on extracted edges and also amplify high frequency noise (see Table 3). These methods result in disconnected edges, too. Only the Canny edge detector of this group leads to connected extracted edges, however in a blind manner, and thus may lead to wrong edges and corners. In region-wise methods, the edges are extracted using a small neighborhood of pixels. Rank-based filters [16], statistical methods, Fourier-transform [17, 18], Spline-interpolation [19], Laplacian-based and wavelet-based [20] are samples of region-wise methods. All of these methods could extract salient and acceptable edges. However, they need heavy preprocessings leading to high computational costs. For example, in the rank-based method, instead of raw pixel values, the value near median of a neighborhood around the edge pixels is selected. In statistical methods, the distribution function of two neighboring objects is used to determine the edge pixels. Fourier transform-based edge extraction methods use the frequency responses of edge pixels. In Spline-interpolation-based methods, first the edge pixels are interpolated along a Spline and then the edge pixels are extracted. In Laplacian- and wavelet-based methods, the edge extraction and its verification is done in a multi resolution manner. As in the proposed algorithm the edge corners are used as the CPs, here we review the existing corner detection algorithms as well. In gray-scale images, corners are characterized by using the second derivatives of the image luminance function [21]. Although this method does not require pre-segmented image contours, it is sensitive to the noise amplification effects of the second-derivative operators. In general, grayscale corner detection algorithms can be divided into two main groups, template-based [22] and gradient-based [23]. In the first category, for each subimage, the similarity between a given template of a specific angle and that subimage is detected. Because of utilizing multiple orientation templates, the technique is computationally expensive. On the other hand, the second category relies on measuring the curvature of an edge that passes through a neighborhood. The strength of the corner detection response depends on both the edge strength and the rate of edge direction change. Gradient-based corner detection techniques are more likely to respond to noise than their contour-based counterparts, and often perform quite poor results. The Harris corner detector [24] uses the auto-correlation matrix to recognize the corners, but this method cannot be applied in multiscale mode. As such, in [25], Harris corner detector is applied on a Gaussian pyramid. In [26], after point symmetry transform of edge pixels in the area of local window by the corner point as the center of symmetry, there is no intersection between the original edge pixels and the point symmetry transformed edge pixels

4

for the corner. On the other hand, there must be at least one intersection point between them for the non corner. After corner candidates are extracted by point symmetry transform of the edge image, distance weight function and phase weight function eliminate false corners for the corner verification and the exact localization of the real corners. Slit rotational edge feature detector (SRED) is another corner detection method which evaluates the edge certainties at each pixel in all directions [27]. Due to the variation of corner detection accuracy with the directions of edges, and also disabilities of SRED in detecting the corners in the regions of low contrast, in [28] weighting and interpolating around the corner pixels are used. 2.2 Feature matching: In this step, the correspondence between the features detected in the reference and sensed images is established. Various feature descriptors (along with spatial relationships among them) and similarity measurements are reported for this purpose. o Feature descriptors include: B-Splines [29], chain code [5], snake-model [30], length code

algorithms [31]. o Similarity measures include: iterative closest points (ICP) [32], Bessel algorithm [33],

geometric ICP [34], least squares error (LSE) [35], normalized cross correlation (NCC) [30], average magnitude difference function (AMDF) [36], and maximization of mutual information (MMI) [37].

2.3 Transform modeling: In this step, the type and parameters of mapping function which aligns the sensed image to the reference image are estimated. The parameters of the mapping function are computed by means of the established feature correspondence [1]. 2.4 Image resampling: In this step, the sensed image is registered by means of the mapping function. Image values in non-integer coordinates are computed by the appropriate interpolation technique. 3. Multiresolution Transforms In this section, a brief overview on real wavelet, dual-tree complex wavelet, curvelet, and contourlet transform is given and their advantages and drawbacks are discussed. 3.1. Real wavelet transform The two-dimensional real wavelet transform decomposes an input image into coarser resolution subimages; which consists of a low frequency (approximation) subimage and three types of high frequency (detail) subimages [38]. However, it suffers from poor directional selectivity, subband mixing problem, and not having the shift-invariant property due to its downsampling stage. A well-known approach for providing shift invariance is to use the undecimated form of the dyadic wavelet transform. However, undecimated wavelet transform suffers from increased computation requirements and high redundancy in the output subimages, causing subsequent processes to become computationally expensive. In fact, the undecimated version of real wavelet transform is shift-invariant but it imposes high redundancy and still suffers from poor directional selectivity and subband mixing problem.

3.2. Dual-tree complex wavelet transform Dual-tree complex wavelet transform was proposed by N. Kingsbury [39] in 1997, to avoid two disadvantages of real wavelet transform; namely, lack of shift invariance and poor directional selectivity. The complex wavelet transform (CWT) has solved the problem of inability of perfect reconstruction and good frequency portioning using complex wavelets.

5

3.3. Curvelet transform Curvelet transform was proposed by Candes and Donoho [40] in 1998, as a new representation which can achieve optimal approximation behavior in a certain sense for 2-D piecewise smooth functions in 2ℜ , where the discontinuity curve are bounded and are twice continuously differentiable. In summary, the curvelet decomposition applies ridgelet transform on nonoverlapping blocks of the input image, where the partitioning size depends on the curvature on input curve. 3.4. Critically sampled contourlet transform Contourlet transform is a multiscale and multidirectional framework of discrete image. The main concept of contourlet transform was proposed by Vertelli in 2001. It was then implemented in 2-D domain by Do and Lu in 2003 as a simple directional extension for wavelet that fixes its subband mixing problem and improves its directionality [41]. In this transform, the multiscale and multidirectional analyses are separated in a serial way (see Fig. 1). The Laplacian pyramid (LP), is first used to capture the point discontinuities followed by a directional filter bank (DFB) to link point discontinuities into linear structures. The LP decomposition at each level generates a downsampled lowpass version of the original data. The process can be iterated on the downsampled signal. The new DFB-based quincunx filter bank (QFB) which avoids the modulation of the input images and has a simpler rule for expanding the decomposition tree is adopted by contourlet transform. The DFB is designed to capture high frequency components of images.

Figure 1. 1-level contourlet decomposition [42].

3.5. Undecimated contourlet transform To achieve the shift-invariance property, the undecimated (nonsubsampled) contourlet transform was built upon nonsubsampled pyramids and nonsubsampled DFB in 2005. The undecimated pyramid is completely different from the Laplacian pyramid in critically sampled contourlet transform. The building block of the undecimated pyramid is a two-channel filter bank which has no downsampling stage, and hence is shift invariant [43]. 3.6. Contourlet against wavelet, complex wavelet and curvelet transforms The main contributions of contourlet transform are: - Good directional selectivity: Directional information (almost like the information which

extracted by CWT and CV) can be captured efficiently when using contourlet. The final result is a double iterated filter bank structure, which decomposes images into directional

HH

LL

Wavelet Transform

HL

LH

0

1

2

3

4

5

6

Wavelet Decomposition Directional Filter Bank

6

subimages at multiple scales. As shown in Fig. 2, WT mixed the diagonal information of input images.

(a) (b)

Figure 2: (a) Input image. (b) 2-D impulse response of real wavelet transform to the input image at level 4 [35].

- Perfect reconstruction: With perfect reconstruction Laplacian pyramid and directional filter

bank, it has perfect reconstruction (almost like CWT). - Noise restraint: Its ability of noise restraint is better than wavelet transform [44]. It can be

designed to be a tight frame, which implies robustness against noise due to quantization or thresholding.

- Extraction of subjectively continues edges: It is using the contour segment-like basis to approximate the original image; and thus can result in connected smooth edges. The extracted edges are more subjectively acceptable than those extracted by CWT and CV.

- Critical sampling: It is almost critically sampled, with a small redundancy factor of up to 1.33, it is much better than CWT and CV.

- Its priority against other multiscale, multidirectional decompositions (such as cortex transform and steerable pyramid) is that it allows different number of directions at each scale while nearly achieving critical sampling.

4. Proposed Method In this paper, an algorithm for registering the panchromatic satellite images is proposed. The main focus is on feature extraction and feature matching steps. The effective contourlet transform is used for feature extraction step. The proposed algorithm is explained as follows: 1- Two input satellite images (reference and sensed images) are decomposed using the

contourlet transform (nlevels = [3 3 3], this shows that decomposition are done in 3 level and in each level 8 different direction filtered).

2- To eliminate weak coefficients (corresponding to noise and weak edges) from highpass subimages, we compute the adaptive threshold

)1(0))),((*)))(min())(((max())(min(,2 =−+<<< coeffsiDIstdiDIiDIiDIcoeffifni

where DI(i) denote highpass coefficients at level i and std denote the related standard

deviation value. 3- To compute the enhanced edge strengthened images, the highpass subimages resulted from

applying eq. 1, and the unchanged lowpass coefficients resulted from step 1 are used to reconstruct the images.

4- Edge extraction process (using Sobel operator) is applied on the reconstructed edge strengthened images resulted from Step 3, and the connected edges, which are longer than a determined threshold (using Eq.2), are represented using a chain code descriptor (see Fig. 4). For chain code representation, the edge map image is scanned from left to right and top to bottom. As such, the edges are represented by an integer sequence }}7,...,2,1,0{{ ∈ia ,

7

which depends on the relative position of the current edge pixel with respect to the previous edge pixel, Fig. 3. It is worth to mention that our experiments showed that, using Eq. 2, we can obtain most of strong edges which are unlikely to change by noise. Also, considering Eq. 2, small changes (maximum 3 length sequences, such as 0, 11, 222 and the forth) between a long sequence of other directions (minimum 3 length sequence at each side) are mostly caused by noise and thus should be discarded. For doing this, the resulting chain code is verified and the undesirable direction changes in edges are ignored. For example, if in the part of a chain code become (777777700777777), the (00) denotes a change in direction which is assumed to be caused by noise and therefore it replaced by (77). As, we need a fast and robust descriptor of extracted edges to apply our corner detection method on it, we adopted the chain code scheme.

Figure 3: Chain code representation. [P is the current pixel.] 5- Corners along edges, obtained from Step 4, are considered as our CPs. For detecting the

corners, here we search in the obtained chain code representation of edges and consider all changes in all direction, except for oo 180,0 directions. Then, as listed in Table 1, the intersection of specific sequences is considered as the corners. Our experimental results show that the proposed corner detection approach works well in remotely-sensed images.

Table 1: Corners considered as the intersection point of listed sequences.

Sequence of

Followed by Sequence of

1’s or 5’s 0’s or 2’s or 3’s or 4’s or 6’s or 7’s 2’s or 6’s 0’s or 1’s or 3’s or 4’s or 5’s or 7’s 3’s or 7’s 0’s or 1’s or 2’s or 4’s or 5’s or 6’s 4’s or 8’s 1’s or 2’s or 3’s or 5’s or 6’s or 7’s

6- For matching the CPs, we compare a small window around each control point in the

reference image with the same size window around each control point in the sensed image. Since the maximum rotation in satellite images is about o45 , then seven o15 rotated windows in the sensed image are compared. Our experiments show that {-45, -30, -15, 0, 15, 30, 45} degrees rotated blocks are sufficient for matching control points. The crosscorrelation value between the search block in the reference image and the corresponding rotated block in the sensed image is then used for measuring the likelihood. In fact, the center of windows with maximum crosscorrelation value indicates the matched control points in both images. It is worth to mention that one important consideration for avoiding incorrect matches is that the transformation in satellite images can be globally modeled by an affine transform. Therefore, in verifying the matched CPs, we only need to compare the rotation degrees of the searched windows.

Threshold = 1/3 longest width or height of original image (2)

123

4

5

P

6

0

7

8

7- We noticed that in satellite images, only translation (up to 1/3 longest edge in image), rotation (up to o45 ), and scaling might have happened. Therefore, using matched CPs, the parameters θ,,, stt yx of the affine transform model between reference and sensed images are estimated using

−+

=

1

1

2

2

cossinsincos

yx

stt

yx

y

x

θθθθ

(3)

where ),( 11 yx and ),( 22 yx are the CP coordinates in the reference and sensed image, respectively, yx tt , are the translation value in x and y direction, s is the scaling factor, and θ is the rotation angle. Using the estimated model parameters, resampling process is applied on the edge map images.

8- The correctness of estimated model parameters is verified using all long edges in both edge map images. If the RMSE between the reference and sensed image edges is lower than a predetermined value, the original sensed image is transformed using the estimated affine transform to be registered with respect to the reference image.

5. Experimental Results We implemented the proposed registration algorithm using Matlab 7.0.4, on a dual core, Pentium 4 computer with 1GB Ram. Our images were obtained from an image library containing the images from different parts of our country (Iran), and also some cities like Banda and Rome. They were taken by IRS and IKONOS satellites. Fig. 4, shows two typical input images; IKONOS image from Booshehr city in Iran. The contourlet decompositions of these images are shown in Fig. 5.

(a)

(b)

Figure 4: IKONOS satellite images from Booshehr city, in Iran: (a) reference image, (b) sensed image.

9

(a)

(b)

Figure 5: Contourlet decomposition of input images shown in Fig. 4: (a) reference image, (b) sensed image.

The reconstructed input images after applying Eq.1 on decomposed highpass subimages are shown in Fig. 6. The corners of continues long image edges which are selected as CPs are shown in Fig. 7. Fig. 8 shows a sample searching windows, to find the matched CPs. Registered images are shown in the Fig. 9.

(a)

(b)

Figure 6: Contourlet reconstructed IKONOS satellite images from Booshehr city, in Iran: (a) reference image, (b) sensed image.

10

(a)

(b)

Figure 7: Extracted corners from obtained edge map images (control points): (a) reference image, (b) sensed image.

Figure 8: Searching for the corners in different directions.

[Block searched window 11×11 is in the reference image and its rotated versions are in the sensed image.]

Figure 9: Registered image, using proposed contourlet-based method.

11

For comparison purposes, we selected three state-of-the-art transforms, real wavelet and dual-tree complex wavelet transform in the decomposition step. We used the built-in wavelet function of Matlab 7.0.4 tool and the library functions of dual-tree complex wavelet [45]. We applied the subsequent steps of our algorithm on a group of sample images. The results are shown in Table 2 and Figures 11-14. Considering the results listed in Table 2, we should mention the following. - Edge map accuracy: In order to measure the accuracy of the obtained edge maps we used a

ground truth edge map. In order to obtain the ground truth edge map of input images, first, the opinion of 3 expert user of satellite images about the important edges of input images which can improve the registration accuracy were collected. Then, using a general derivative operator (Canny) and morphological filtering (dilation and erosion), the selected edges in the previous step were obtained. The resulted edge map was considered as the ground truth. For each image, the distance between the ground truth edge map and the obtained edge map was considered to measure the accuracy of extracted edges, using

2int

2int )()(

&int

GTedgepoGTedgepo yyxxD

transformContourletCWTbyextractedpoedge

−−−=

∀ (4)

As listed in Column 2 of Table 2, when we used the contourlet-based method the distance

between the two edge maps decreased; and therefore subsequently the accuracy of the CPs increased leading to better registration results. Depending to the quality and contrast of input images, we obtained slightly different results. The ranges of the results are shown in Column 1 of Table 2.

- Registered image accuracy (pixels): In order to measure the accuracy of the registration process, we used the image division operator. The division operator is one of the change detection approaches which can eliminate the intensity changes in the input images. Also, we compared the registration accuracy before and after applying the enhancement process on the edge map. The improvement was about %27 on the resultant registration accuracy.

- Registration error: The registration error affects the change map image. Depending to the quality and accuracy of extracted edges (and thus the efficiency of CPs), we obtained slightly different results. The ranges of the results are shown in Column 2 of Table 2. As can be seen in this table, the accuracy of 0.2 pixels is obtained; which is a suitable range for the registration accuracy between the reference and sensed images in the change detection applications.

- Execution time: Considering the computational cost, the wavelet transform performs faster than the contourlet and complex wavelet transform. But, as can be seen from Fig. 11, as it performs in point-wise manner it produces isolated edge points (and not continuous edge lines). Also, the redundancy and imaginary results of the CWT makes it time consuming (more than 5 times more than the WT and 3 times more than the CT). The related elapsed times are given for the 512×512 pixel images.

- Line extraction: Considering the continuity of extracted edges, we compared the results obtained by WT-, CT- and CWT-based methods. As shown in Fig. 12, the WT-based method produces isolated points; and thus we cannot apply it for corner detection purposes. But, the CT- and CWT-based methods resulted in continuous edge lines.

- Subjectively acceptable extracted edges: Contourlet-based method resulted in extracted edges which were subjectively more acceptable that the CWT-based method.

12

Table 2. Performance analysis of different multiresolution methods. Transform

Method Edge Map Accuracy

Registered Image

Accuracy (pixels)

Execution Time

(mins)

Line Extraction

Subjectively Acceptable Edges

Real Wavelet 0.42 - 0.32 0.42 - 0.33 4 - - Undecimated Real Wavelet

0.36-0.18 0.31-0.25 6 short lines not bad

Curvelet 0.35-0.25 0.25 - 0.2 10 good good DT Complex

Wavelet 0.34 - 0.025 0.23 - 0.18 20 good good

Contourlet 0.018 - 0.013 0.17 - 0.14 6 good very good

Table 3, shows the comparison results of the proposed edge enhancement method by the existing ones. In order to compare these methods, the following measures are used: - Ringing effect: in order to measure the quality of input image edges, we consider the ringing

effect which is cause by the reviewed methods. - Noise amplification: as it showed in Column 2 of Table 3, all of point-wise operators strength

the noise, however this is removed in applying region-wise operators. - Extract continues edges: as extracting the most salient edges of the image is needed, in order

to obtain the distinguishable corners, we used this measure to evaluate edge enhancement methods. After applying these methods on different images, we obtained the results listed in Column 3 of Table 3.

- Computational cost: in this part, the computations which are needed in order to improve the image edges are grouped in three categories; low, medium and high.

Table 3. Comparison results of different edge extraction methods.

Edge Extraction Methods Ringing Effect

Noise Amplification

Extract Continues Edges

Computational Cost

High and Bandpass Filters

Caused Caused No Low

Sobel, Robert and Prewitt

Caused Caused No Low

Point-Wise

Canny Caused Caused Blind Continues Medium Rank-Based --- --- Almost

Continues Medium

Statistical-Based --- --- Almost Continues

High

Fourier-Based --- --- Almost Continues

High

Spline-Interpolation --- --- Good High Laplacian-Based --- --- No Medium Wavelet-Based --- --- No Medium Undecimated

Wavelet-Based --- --- Almost

Continues Medium

DT Complex Wavelet-Based

--- --- Good High

Curvelet-Based --- --- Good Medium

Region-Wise

Cotourlet-Based --- --- Excellent Medium

13

As we needed more continues edges in order to extract the salient corners as the CPs, the contourlet transform with a medium computational cost lead us to the best extracted continues edges in comparison with other existing methods. Fig. 10 shows the edge map of input images without applying any edge enhancement method. In this part, we analyze the edge map of applying simple Sobel and Canny operator against the wavelet, complex wavelet, and contourlet with/without edge enhancement process; in which small contourlet coefficients were discarded. As shown in Fig. 10, Sobel operator could not extract connected long edges from images, however, Canny operator spoiled the real edges by linking them together. Fig. 10c-e shows the edge maps of lowpass band of the input image, after applying 3 levels of decimate and undecimated wavelet, complex wavelet and contourlet. Decimated wavelet could not extract continues edges, however, undecimated wavelet extracted longer edges than the decimated one, but they are not strong enough. Complex wavelet extracted too detailed images. By discarding small coefficients of contourlet decomposition (shown in Fig. 10g), we could improve the edge map (shown in Fig. 10f). Eliminating the weak edges in the corner detection step, led us to achieve better CPs, resulting in better registration accuracy fro about %27. Table 4, lists the comparison results of the proposed corner detection method with the existing ones. In order to compare these methods, the following measures were used: - Sensitivity to noise: in order to assess this measure, we applied the methods on different

images with variable amounts of noise. The average percent of corners which were lost due to noise are listed in Column 1.

- Computational complexity: in this part, the required computations to detect the corners were grouped in three categories; low, medium, and high.

- Sensitivity to image contrast: in order to assess this measure, we applied the methods on different images with variable contrasts. The average percent of corners which were lost by decreasing the contrast of images are listed in Column 3.

- Other dependencies of methods: in this evaluation, we reported other dependencies of these methods.

14

Sobel Operator

Canny Operator

Real Wavelet Transform

(a) (b) (c)

Uudecimated Real Wavelet Transform

DT Complex Wavelet Transform

Curvelet Transform

(d) (e) (f) Contourlet Transform

Enhanced Contourlet Transform

(g) (h)

Figure 10: Extracted image edges from the reference image, using: (a) Sobel, (b) Canny, (c) decimated real wavelet,

(d) undecimated real wavelet, (e) DT complex wavelet, (f) curvelet, (g) contourlet, (h) enhanced contourlet.

15

Table 4. Comparison results of different corner detection methods. Method Sensitivity to

noise Computational

Complexity Sensitivity to

Image Contrast Other Dependencies

Template-Based <%25 high <%25 --- Gradient-Based >%65 middle >%30 & <%50 ---

Point Symmetry Function (Harris)

<%60 & >%30

middle >%32 & <%53 Existing at least one intersection point

Slit Rotational Edge Detector

<%54 & >%20

high >%20 & <%65

Good interpolation around the edge pixels

Proposed Chain Code-Based <%25 low <%35 ---

As shown in the obtained results, by a simple chain code descriptor and thus with a low computational cost, we extracted salient corners and decreased the sensitivity to noise and image contrast. It should be noted that if there be no distinguishable corners or if the matched corners be corrupted in the sensed images, the algorithm will cause some registration error. Figure 11-14b, show the images in which in the first glance the matched corners can be perceived by the user, but due to the disturbance of corner points (due to shadows in the sensed image), a registration error of about 3 pixels is obtained. Also, the denoising method using chain code representation (discussed in Section 3), might discard some of the corners appeared in the form of ..7777007777.. or ..22223332222… But, using the maximum 3 length of them, it is not much probable. However, our experiments on a large number of images showed that concerning the number of eliminated CPs and the denoising effect made in edges, it works efficiently. 6. Conclusion In this paper, an automatic contourlet-based edge extraction method for registration purposes was proposed. It was shown that as contourlet transform can extract strong and continuous edges from images, it can be effectively used in the feature extraction step of registration applications. As a preprocessing step, we used the contourlet transform to discard weak edges which might lead the matching step to detect wrong matched points. Also, we developed an approach for finding the edge corners and denoising the extracted edges using the chain code representation. The experimental results showed the efficiency of the proposed method when compared to other available approaches. Acknowledgement This work was in part supported by a grant from ITRC.

16

Reconstructed reference image edges using real wavelet transform

Reconstructed sensed image edges using real wavelet transform

(a)

Reconstructed reference image edges using complex wavelet transform

Reconstructed sensed image edges using complex wavelet transform

(b)

Reconstructed reference image edges using contourlet transform

Reconstructed sensed image edges using contourlet transform

(c)

Figure 11: Extracted edges after applying: (a) real wavelet transform , (b) dual-tree complex wavelet transform, (c) contourlet transform.

17

(a)

(b)

(c)

Figure 12: Sample input images capture in two time instances: (a) Prague 2002, (b) Natanz 2002-2004, (c) Bam 2003-2004.

18

(a)

(b)

(c)

Figure 13: Matched points in sample input images: (a) Prague 2002, (b) Natanz 2002-2004, (c) Bam 2003-2004.

19

(a) (b)

(c)

Figure 14: Registered sample input images: (a) Prague 2002, (b) Natanz 2002-2004, (c) Bam 2003-2004.

References [1] B. Zitova and J. Flusser, “Image registration methods: A survey,” Journal of image and vision ELSEVIER, vol. 21, pp. 977-1000, June 2003. [2] R. J. Radke and et al., “Image change detection algorithms: a systematic survey,” IEEE transaction on image processing, vol. 14, no. 3, March 2005. [3] T. Makela and et al.,”A review of cardiac image registration methods,” IEEE transaction on medical imaging, vol. 21, no. 9, September 2002. [4] F. Eugenio, and et al., “A contour-based approach to automatic and accurate registration of multitemporal and multisensor satellite imagery,” IEEE international geoscience and remote sensing symposium, vol. 6, pp. 3390-3392, Spain, 2002. [5] X. Dai and et al., “Automatic image registration for change detection from Landsat thematic mapper imagery,” IEEE international geoscience and remote sensing symposium, vol. 3, pp. 1553-1555, May 1996. [6] A. Goshtasby and G. C. Stockman,m “Point pattern matching using convex hull edges,” IEEE transaction on systems, Man and Cybernetics 15, pp, 631-637, 1985. [7] M. Holm, “Towards automatic rectification of satellite images using feature based matching,” Proceeding of the international geoscience and remote sensing symposium IGARSS’ 91, Espoo, pp. 2439-2442, Finland, 1991.

20

[8] R. A. Baggs and et al., “Image registration using the length code algorithm,” IEEE transaction on science and technology, pp. 11-14, April 1996. [9] U. Dhond and J. Aggarwal, “Structure from stereo – A Review,” IEEE transaction on system, man and cyber., vol. 19, No. 6, 1989. [10] F. Eugenio, and et al., “A contour-based approach to automatic and accurate registration of multitemporal and multisensor satellite imagery,” IEEE international geoscience and remote sensing symposium, vol. 6, pp. 3390-3392, Spain, 2002. [11] S. Z. Li and et al., “Matching and recognition of road networks from aerial images,” Proceeding of the second European conference on computer vision ECCV’92, pp. 857-861, Italy, 1992. [12] N. Vujovic and D. Bzakovic, “Establishing the correspondence between control points in pair of mammographic images,” IEEE transaction on image processing vol. 6, pp, 1388-1399, 1997. [13] M. Holm, “Towards automatic rectification of satellite images using feature based matching,” Proceeding of the international geoscience and remote sensing symposium IGARSS’ 91, Espoo, pp. 2439-2442, Finland, 1991. [14] J. Ton, A. K. Jain, “Registering landsat images by point matching,” IEEE transactions on geoscience and remote sensing 27, pp 645-651, 1989. [15] M. Sester and et. al, “Definition of ground control features fro image registration using GIS data,” Proceeding of the Symposium on object recognition and scene classification from multispectral and multisensor pixels, pp. 7, 1998. [16] Akaroa, "Colour edge enhancement," Proceedings of. Image and Vision conference, pp. 297-302, 2004. [17] M. Asif khan, "Analysis of edge enhancement operators and their application to SPOT data," International journal of remote sensing, vol. 13, pp. 3189-3203, 1992. [18] B. Chen and et. al, "Edge enhancement of remote sensing image data in the DCT domain," Proceeding of SPIE, vol. 3387, pp. 313-323, 1998. [19] Z. Peifeng and et. al, "Image interpolation with edge enhancement," Information processing society of Japan, vol. 2001, pp. 27-32, 2001. [20] Y. Sun and et. al, "Improvement of ultrasound image based on wavelet transform: speckle reduction and edge enhancement," IEEE transaction on medical imaging, , vol. 25, pp. 297-311, 2006. [21] D. M. Tsai. Boundary-based corner detection using neural networks. Pattern Recognition, 30:85–97, 1997. [22] R. Mehrotr, S. Nichani, and N. Ranganathan. Corner detection. Pattern Recognition, vol. 23, pp. 1223–1233, 1990. [23] L. Kitchen and A. Rosenfeld. Gray-level corner detection. Pattern Recognition Letters, vol. 1, pp. 95– 102, 1982. [24] Harris and et al., “A combined corner and edge detector,” proceeding of the 4th Alvey vision conference, pp. 147-151, 1988. [25] F. Zhao and et al., “Image matching by multiscale oriented corner correlation,” ACCV, Springer-Verlag Berlin , LNCS 3851, pp. 928-937, 2006. [26] E. J. Park and et al., “Point symmetry based corner detector with distance and phase weight functions,” proceeding of signal processing, pattern recognition, and application, vol. 520, 2006. [27] Y. Etou and et al., “Corner detection based on edge and geometry information,” proceeding of IEICE general conference, D-12-7, 2001. [28] Y. Etou and et al. “Corner detection using slit rotational edge-feature detector,” IEEE international conference on image processing , vol. 2, pp. 797-800, 2002. [29] Z. Peifeng and et al., "Image interpolation with edge enhancement," Information processing society of Japan, vol. 2001, pp. 27-32, 2001. [30] W. Peckar, “Two-step parameter free elastic image registration with prescribed point displacement,” Lecture notes in computer science, vol. 1310, proceedings of 9th international conference on image analysis and processing, pp. 527-534, 1997, London. [31] R. A. Baggs and et al., “Image registration using the length code algorithm,” IEEE transaction on science and technology, pp. 11-14, April 1996. [32] G. C. Sarp and et al., “ICP registration using invariant features,” IEEE transaction on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 90-102, January 2002. [33 ] D. Ionexcu and et al., “A registration and matching method for remote sensing images,” IEEE conference on electrical and computer engineering, vol. 2, pp. 14-17, 1993.

21

[34] Y. Liu and et al., “Accurate registration of structured data using two overlapping range images,” IEEE transaction on robotics and automation, vol. 3, pp. 2519-2524, May 2002. [35] S. Umeyama, “Least-squares estimation of transformation parameter between two point patterns,” IEEE transaction on .pattern analysis and machine intelligence, vol. 13, pp.376-380, April. 1991. [36 ] D. I. Barnea and et al., “A class of algorithms for fast digital image registration,” IEEE transaction o computing., vol. C-21, pp. 179-186, February 1972. [37] F. Maes and et al., “Multimodality image registration by maximization of mutual information,” IEEE transaction on medical imaging, vol. 16, no. 1, april 1997. [38] X. Yang, W. Yang, and J. Pei, “Multi- focus images fusion based on wavelet decomposition and evolutionary strategy,” Proceeding of the 2003 international conference on neural networks and signal processing, vol. 2, pp. 951-955, December 2003. [39 ] N. Kingsbury, “The dual-tree complex wavelet transform: a new efficient tool for image restoration and enhancement,” European signal processing conference, pp.319-322, 2006. [40 ] E. J. Candes and et al., “Curvelet – a surprisingly effective nonadaptive representation for objects with edges,” In A. Cohen and L. L. Schumaker, editors, Curve and Surface Fitting, Saint-Malo, 1999. Vandetbilt University Press. [41] M. N. Do. “Directional multiresolution image representations,” PhD thesis, Swiss federal institute of technology, Lausanne, Switzerland, December 2001. [42] M. Qiguang and et al., “A novel image fusion method using contourlet transform,” IEEE transaction on communication, circuits and system processing, vol. 1, pp. 548-552, June, 2006. [43] Zhou, A. L. da Cunha, and M. N. Do, “Nonsubsampled contourlet transform: construction and application in enhancement,” Proceeding of IEEE International Conference on Image Processing, Genoa, Italy, September 2005. [44 ] M. N. Do and et al., “The contourlet transform: An efficient directional multi resolution image representation,” IEEE transaction on image processing, vol. 14, pp. 2091-2106, 2005. [45 ] http://www-sigproc.eng.cam.ac.uk/~ngk/