GISdevelopment.net ---> AARS ---> ACRS 1999 ---> Poster Session 1 Initialization for Image Registration using Feature Matching Liang-Chien Chen and Jeng-Daw Lee Center for Space and Remote Sensing Research National Central University, Chung-Li Tel: (886)-3-4257232 , Fax: (886)-3-4254908 E-mail: lcchen@csrsr.ncu.edu.tw China Taipei Keywords: Image Registration, Segmentation, Feature Matching Abstract We present here a strategy to initialize the registration using feature matching. The work includes extraction for feature polygons, description for region boundaries, and similarity assessment. We combine three descriptors for feature polygons, namely, Shape-Matrix, Fourier Descriptors, and Invariant Moments in the matching scheme. Three descriptors are both scale and rotation invariant. Matching conjugate polygons with Shape Matrix approach is reliable only when the principal axis is unambiguous. Fourier Descriptor is good at detail descriptions. Invariant Moments are suitable to correspond the polygons with smooth boundaries. Combining those complementary descriptors in similarity assessment, we propose a selecting scheme to generate reliable matching pairs. Experimental results indicate that the matching between an airborne scanner image and an aerial photo is reliable. Introduction The registration between a reference image and its counterpart, a second remotely sensed image, is a necessity in many image analysis tasks such as change detection, feature or color enhancement, map revision, and data fusion. Two approaches are possible. The first is rigorous orthorectification [Mayr & Heipke, 1988]. Through orthorectification for each image, multi-temporal and multi-source images are co-registered in the ground coordinate system. The approach is rigorous and robust. However, it needs orientation parameters for the sensor in addition to a digital terrain model (DTM). The second approach, on the other hand, performs image-to-image registration [Goshtasby et al., 1986]. This approach does not require orientation parameters or DTMs. However, a reference image is needed. Considering the advantages of the second approach, we will focus our investigation on the image-to-image registration. The procedure of image registration may be divided into two steps. The first is to select enough registration control points (RCPs) then to measure the corresponding image coordinates. The second step is to choose a mapping function after which a coordinate transformation is performed. The first step is essentially the key work in automated registration. Several approaches for automating the procedure have been proposed [Goshtasby et al., 1986; Nevatia &Medioni, 1984]. Those approaches suffer from the following limitations: (1) the number of RCPs is often not sufficient, (2) the distribution of RCPs is not always uniform, and (3) the point-to-point correspondence is not always sufficiently accurate. To cope with the weaknesses, Chen & Lee [1992] proposed a scheme to densitify the control frameworks. The method was also successfully implemented in registering an airborne scanner image on a digitized aerial photo [Chen & Rau, 1993]. One weak point of the method is that at least 3 RCPs are needed to provide the initial registration. We, thus, propose here a scheme to perform feature matching for initializing the registration. The proposed scheme includes three major components: (1) feature extraction, (2) feature description, and (3) similarity assessment and image matching. Feature Extraction Points, lines, and polygons are the three types of image features. Considering the content of shape information, which is crucial in feature matching, we select shape polygons for processing. Image segmentation is an essential procedure to extract feature polygons from images. In the segmentation, we use “Energy” [Pratt, 1991] of gray values as feature index to segment an image. To improve the results of segmentation, a smoothing preprocess is preferable. In order to preserve edge information in a smoothing procedure, we combine two methods to achieve that. Those methods include (1) Adaptive Smoothing (AS) [Saint-Marc et al., 1991] and (2) Symmetric Nearest Neighbor filter (SNN) [Harwood, et al., 1987]. The combination of the methods achieves a goal that each segmented block is more homogeneous while the edges are still preserved. To further enhance the edges, we consider the Multi-resolution Edge Detection (MEDT) [Deok, 1995] method to strengthen the edge effect. After calculation the edge strengths, which are normalized from 0 to 1, we multiply the grey values by the strength values to enhance the block boundaries. Finally, “Energy” value is computed as a segmentation index. Feature Description Three feature measurements are considered namely, Shape Matrix (SM) [Flusser, 1992], Fourier Descriptor (FD) [Pratt, 1991], and Invariant Moments (IM) [Pratt, 1991]. Three descriptors are both scale and rotation invariant. Matching conjugate polygons with SM is reliable only when the principal axis in unambiguous. FD is good at detail descriptions. IM is suitable to correspond those polygons with smooth boundaries. Considering the complementary characteristics, three measurements are combines in further matching procedure. Similarity Assessment and Matching We describe the three indices for three descriptors for measuring the shape similarity. Then a matching strategy for combining three indicators will be provided. Similarity Assessment We use root mean square difference (RMSD) to measure the similarity for FD and IM. For a sensed image and its counterpart, i.e., reference image, the RMSD for FD is defined as br> Two shapes are with higher similarity when smaller RMSD is observed. The RMSD for seven IMs is defined as For SM, referring to figure 1, the similarity between shape A and B is calculated as Figure 1. Illustration of SimilarityAssessment for SM The index P(A,B) reflects higher similarity when higher value is calculated. Matching Considering three indices, i.e., RMSDFD, RMSDIM, and P(A,B), a successful matching should fulfill following criteria: (1) RMSDFD<=dFD (2) RMSDIM<=dIM (3) Min(RMSDFDxRMSDIM (4)P(A,B)<=dP(AB),Max(P(A,B)) (5)A subset of the intersection from the three sets that fulfills the four criteria is selected as the potential matching pair.] Experimental Results The test area is located in north Taiwan. The reference image is an orthorectified aerial photo with 2000x2000 pixels at 1.12m pixel spacing as shown in figure 2(a). The sensed image is an airborne scanner image with 512x512 pixels at 4.5 nominal ground resolution as shown in figure 2(b). The sensed image was resampled to 2000x2000 pixels for processing convenience. The preprocessed images, including the procedure of AS, SNN and edge enhancement, are shown in figure 3. The segmented blocks using Energy method are shown in figure 4. (a) (b) Figure 2. Test Images (a) Reference Image (b) Sensed Image (a) (b) Figure 3.Preprocessed Images (a) Reference Image (b) Sensed Image By visual inspection, the corresponding polygon pairs are shown in table 1. Considering the combined results of FD and IM, table 2 shows the correspondence. The correspondence using SM only is illustrated in table 3. Combining FD, IM, and SM, the matching results are shown in table 4. The final results, with robust estimation, are shown in table 5. (a) (b) Figure 7. Segmented Polygons with IDs (a) Reference Image (b) Sensed Image Table 1. Corresponded Polygons Sensed Reference s1 r1 s3 r3 s4 r4 s5 r5 s6 r6 s8 r7 s10 r8 s11 r9 s12 r10 s13 r13 s14 r14 s15 r16 s16 r15 s17 r17 s18 r18 s20 r20 Table 2. Combined Correspondence for FD & IM Sensed Reference s1 r1 s4 r2 s5 r4 s6 r12 s8 r7 s10 r15 s11 r9 s12 r10 s13 r13 s15 r16 s17 r17 s20 r20 Table 3. Correspondence by SM Sensed Reference s1 r1 s3 r3 s5 r5 s6 r6 s8 r7 s10 r8 s12 r10 s15 r16 s16 r15 s17 r17 s18 r18 s19 r14 s20 r20 Table 4. Matching Results Table 5. Robust Estimated Results Sensed Reference s1 r1 s8 r7 s12 r10 s15 r16 s17 r17 s20 r20 Sensed Reference s1 r1 s8 r7 s12 r10 s15 r16 s17 r17 s20 r20 It is observed that if only FD and IM are used, more polygon pairs will be selected. While some erroneous correspondence (S4, S5, S6, and S10) are remained. On the other hand, if we use SM, the situation is similar. The difference is that the remained erroneous corresponding polygon (S19) is different. When three descriptors are combined, the selected polygon pairs are less while with highest reliability. It is the purpose of this investigation that we only need small amount, 3 for instance, of RCPs for initialization of image registration. Thus, the proposed scheme is validated for the time being. Concluding Remarks The experimental results indicate that the proposed scheme may select RCP pairs with very high reliability. Although the number of selected pairs is less, the reliability is our concerned. Thus, the test is successful. However, we still need further tests to ascertain the applicability. It should be pointed out that the proposed scheme might only be used for the area of rolling terrain. For those images with rugged terrain or large scale images with high-rises the scheme may result unreliably. References Chen, L.C., and Lee, L. H., 1992, Progressive Generation of Control Frameworks for Image Registration, Photogrammetric Engineering and Remote Sensing, 58 (9), pp.1321-1328. Chen, L. C., and Rau, J. Y., 1993. “Geometric Correction of Airborne Scanner Imagery Using Orthophotos and Triangulated Feature Points”, International Journal of Remote Sensing. 14 (16), pp.3041-3059. Deok, J. P., Nam, K. M., and Park, R. H., 1995, Multiresoultion edge detection techniques, Pattern Recognition, 28 (2), pp. 211-229. Flusser, J., 1992. Invariant shape description and measure of object similarity, Proc. 4th Internat. Conf. Image Processing, Maastricht, The Netherlands, pp.139-142. Goshtasby, A., Stockman, G., and Page, C., 1986. A region-based approach to digital image registration with subpixel accuracy, IEEE Trans. On Geoscience and Remote Sensing, 24 (3), pp. 390-399. Harwood, D., Subbarao, M., Hakalahti, H., and Davis, L. S., 1987, A new class of edge-preserving smoothing filters, Pattern Recognition Letters, 6, pp. 155-162. Mayr, W. and Heipke, C., 1988, A contribution to digital orthophoto generation, International Archives of Photogrammetry and Remote Sensing, Kyoto, Vol. XXVII, Part B11, pp. 430-439. Nevatia, R., and Medioni, G., 1984, Matching images using linear features, IEEE Trans. on PAMI, 6 (6), pp. 675-685. Saint-Marc, P., Chen, J. S., and Medioni, G., 1991, Adaptive smoothing: a general tool for early vision, IEEE Pattern Analysis and Machine Intelligence, 13 (6), pp.514-529. Pratt, W. K., 1991, Digital Image Processing, 2nd edition, John Wiley & Sons, New York, pp. 563-648.