band alignment and orthorectificatio n of satellite images ... · geomatics indaba proceedings 2015...

Geomatics Indaba Proceedings 2015 – Stream 2

148

Band alignment and orthorectification of satellite images with insufficient attitude- and metadata

by Philip Bouwer, Thinus Prinsloo and Andreas Rademeyer, Pinkmatter Solutions

Abstract

Some remotely sensed satellite images have little or no geometric metadata, e.g. attitude, ephemeris, and tie-point data, making it difficult to produce geometrically accurate imagery. Examples of such imagery are those obtained from RapidEye and SumbandilaSat and are similarly observed with data from many low-cost earth observation satellites. In addition to the lack of metadata, there may be a substantial disparity between the acquisition times of the individual bands. These time differences mean that each band is acquired at a different viewing angle, making band alignment a non-trivial exercise. This paper presents a robust method to generate per-band translation-maps to band align and orthorectify such raw datasets in a single resampling step. A generic sensor-model is constructed using multiple simple polynomials, which are calculated using a reference image, DEM and four initial corner tie-points as ancillary data. The first step is to select a fundamental band, then using the four initial tie-points, project it to some target projection to create a projection translation map. The disparity between the reference image and fundamental band is used in conjunction with a DEM to generate an orthorectification translation map. The projection and orthorectification translation maps are combined into a single translation map that can be used to orthorectify raw image pixels. The inverse of the combined translation map is used to generate a pseudo DEM which overlaps the unprojected (raw) fundamental band. Using this DEM, all other unprojected bands are band aligned to the fundamental band, resulting in a translation map for each band. Each of these translation maps is combined with the orthorectification translation map to form a new band specific translation map. These maps make it possible to band align and orthorectify the unprojected bands in one resampling step. An analysis of the geometric accuracy obtained using this method is presented for RapidEye imagery.

Keywords

band alignment, orthorectification, polynomial geometric senor model, micro-satellites, RapidEye, SumbandilaSat

Introduction

As micro-satellites and other low-cost space borne earth imaging approaches gain popularity, robust mechanisms for image correction become more and more important. Small, low-cost satellites typically exhibit far inferior stability, sensor alignment and sensor geometric calibration than their high-end counterparts. To make the situation even more difficult, low-cost satellites typically record very little attitude data.

Band alignment and orthorectification of remotely sensed optical satellite data is accomplished through geometric correction whereby the image pixels are repositioned to minimise their geolocation error. Having a correctly band aligned dataset is essential for many post-processing systems, such as pixel-based classification systems. Inferior band alignment results in incorrect classifications, as pixels representing different physical locations are compared. In addition, having correctly orthorectified datasets is a requirement of many applications, such as mosaicking and time series analysis.

Many different geometric correction techniques exist and are often specific to a particular satellite. One popular approach is to use generic sensor models such as rational polynomial coefficients (RPC).

This paper presents an extension on RPC usage, where multiple translation maps generated from simple RPC polynomials are combined to generate a generic sensor model for push broom sensors to band align and orthorectify satellite images from low-cost satellites with insufficient metadata.

Methodology

Fig. 1 shows the seven steps required to band align and orthorectify the unprojected bands of an input image. Note that bands are referred to as being part of the input data that needs to be geometrically corrected, in contrast to images which are auxiliary inputs used to help processing steps. The bands are orthorectified using ground control points (GCPs) obtained from a reference image.


149

Projection (polynomial fit)

Unprojected fundamental band Fundamental band corner tie-points &Target projection parameters

Orthorectification

Pseudo DEM generation

Projection translation map

Merged translation map

DEM

Pseudo DEM

Translation map merging

Orthorectification translation map

Projected fundamental band

Target projection parameters

Bfund

Bproj

Mcomb

Dpseudo

Mproj

Mortho

Step 1

Step 2

Step 3

Step 4

Reference imagesIref

DEM

Translation map merging

Merged translation map

Band alignment

Band-alignment translation map per band

Final combined translation map per band

Resample

All unprojected bands

Mf,b Bunproj,b

Malign,bMcomb

Step 5

Step 6

Step 7

Pseudo DEM

Dpseudo

Unprojected fundamental band

Bfund

Unprojected bands

Bunproj,b

Band-aligned and orthorectified bands

Bf

Fig. 1: Logical procedure of the band alignment and orthorectification system.

Each processing step is described in more detail below.

Step 1: Target projection

The first step is to select one of the unprojected bands as the fundamental band. Preferably the fundamental band should be close to nadir, and radiometrically close to the reference image used for ground control. All other bands will be band aligned to this selected band at a later stage. The fundamental band is projected to a target projection using four corner tie-points and the target projection parameters. The target projection parameters specify the projection and ground sampling distance (GSD) (for example UTM-S zone 35, with a GSD of 30 m per pixel). The tie-points must reflect the four corners of the fundamental band in latitude/longitude. This projection is a polynomial fit of the form ‘ ’, and is solved using least squares to obtain coefficients for and .

(1)

(2)

where:

vectors containing the tie-point pixel coordinates

vectors containing the tie-point latitude/longitude coordinates

The extent of the tie-points determines the extent required of the reference image. The corners of the projected fundamental band are calculated using the unprojected band dimensions and the distance between the four initial tie-points. The approximate ground sampling distance per pixel is also calculated:

(3)


150

(4)

where:

(5)

= the Euclidian distance function

the tie-points

The calculated polynomials and GSD can be used to obtain the coordinate for each pixel in geographic latitude and longitude degrees. It is assumed that the GSD does not change over the extent of the data (a false assumption that will be rectified in the orthorectification step). Starting from the upper left tie-point (origin) down to the lower right, in steps of the approximate GSD, each pixel’s coordinate is calculated and projected using a projection library (Proj. 4 [1] or similar). This results in a translation map that describes the offset of each projected output pixel relative to its original location in the unprojected band. This map can be used to resample or translate each unprojected pixel to the target projection. This translation map and the output pixels are saved for later use and henceforth referred to as and respectively.

(6)

where:

= the projection function from target projection to geographical degrees

and:

(7)

(8)

(9)

(10)

with:

& the two polynomials

the tie-points


151

Fig. 2 shows the input and output band for step 1.

Fig. 2: Projection of the fundamental band. Left: unprojected band ( ). Right: projected band ( ).

Step 2: Orthorectification

This step’s goal is to orthorectify the projected output from step 1. Using the extents from , a reference image covering the same area is selected. The reference image is projected and scaled to match the target projection and GSD of . The orthorectification process requires GCPs in order to generate a translation map which in turn can be applied to orthorectify .

The GCPs are obtained by extracting feature points from and using phase correlation [2] to match the features to the applicable band of the reference image. To reduce the search area for a matching feature point in the reference image, a pyramid representation of the image is used to first search on the largest GSD pyramid level. If a good correlation is found, the next pyramid level is searched. This is repeated until the feature point is found on the last pyramid level. If at any stage a poor correlation is found the feature point is discarded.

Each feature point’s search area is adjusted independently and the overall shift of the pyramid level is not accounted for. This ensures that consecutive pyramid level search tiles are centred on the correlation point. This approach is useful for finding feature points where the warp across the extent of the band is irregular. Adjusting all search areas to the mean overall shift would introduce a bias. If this bias is large, it could potentially cause the search tiles not to contain the actual features of interest. Fig. 3 shows an overlay of two image pairs ( and the reference image) with a larger displacement on the left side and smaller displacement on the right.

Fig. 3: Irregular spatial disparity between (green) and the reference image band (red).


152

Fig. 4: Overlay of the reference image (red) and fundamental band (green) before orthorectification (left) and after (right). Middle: enlarged section of the area. Bottom: GCP residuals before and after orthorectification.

The obtained tie-points describe the pixel coordinate shift between the fundamental band and reference image. Since the projection is known, the elevation of each tie-point can be found using a supplied DEM. This results in a list of GCPs, of the form:

(11)

where:

fundamental band tie-points

reference image tie-points and elevation


153

The GCPs are used to generate a RPC geometric sensor model [3]. The RPC model in turn is used to generate a translation map that describes the offset of each orthorectified pixel relative to its input pixel. This process can be repeated several times until the required residuals are obtained. Fig. 4 shows the effect of orthorectification.

It is worth noting that orthorectification will not be successful (the desired residual cannot be met) if the fundamental band and reference image differ to a large degree, for example if the fundamental band has a large cloud cover percentage. The feature point correlation will fail to produce accurate tie-points, which in turn will produce a poor RPC.

Fig. 5 shows the effective translation introduced by the resulting orthorectification map. This data represents the required translation to orthorectify the fundamental band in Fig. 4.

Fig. 5: Visual representation of the orthorectification translation map. Left: heat map for horizontal shift expressed in pixel offset. Right: heat map for vertical shift.

Step 3: Translation map merging, projection & orthorectification

To ensure that the input pixel data is only resampled once in producing the final output product, the translation maps generated in steps 1 and 2 are merged instead of applying two translation steps to the input. Both the X and Y translation components of the projection translation map ( are resampled with the orthorectification translation map ( . Cubic-interpolation [4] is used as the resample method. The resulting combined translation map ( , can be used to warp the unprojected fundamental band directly to an orthorectified fundamental band.

(12)

with:

(13)

where:

the resample function

the merging function

Step 4: Pseudo DEM generation

Band alignment is executed on all unprojected bands. Each band is aligned to the unprojected fundamental band. Normally for this process, an RPC can be used; however, no height values can accurately be obtained for the GCPs, since the unprojected fundamental band will not overlay the supplied DEM well. Alternatively, the unprojected bands can be aligned to the orthorectified version of the fundamental band, but will produce inferior results since the orthorectified fundamental band has already been warped making the correlation less successful. Instead, a pseudo DEM is generated using the inverse of the translation map from step 3. The inverse translation map can be used to warp any orthorectified data back into an unprojected state. This inverse


154

translation map will have the same dimensions as the unprojected fundamental band, making it possible to create an unprojected DEM ( that matches the unprojected fundamental band pixel to pixel.

(14)

where:

projection function from the target projection to latitude ,longitude

original DEM

indexing function (rounded to the nearest integer) – which may cause voids in the DEM.

target GSD components

target origin components

The generated pseudo DEM may have voids, and is void filled using Shepard’s inverse distance weighted (IDW) interpolation function [5]. The filling algorithm uses a distance weighted average method, where the height values of the four nearest non-void pixels (above, below, left and right) are weighted based on how far they are from the void pixel being filled. A void will remain a void if the distance to any of the four non-voids is more than 40 pixels. This ensures that actual voids on the original DEM are not interpolated excessively, which would have resulted in a poorly filled DEM. The only exception to the above procedure is if there are three direct (non-void) neighbouring pixels – in which case the void pixel is filled using the average of the three non-void neighbours. Fig. 6 illustrates the results for generating a pseudo DEM with and without void filling.

Fig. 6: DEM void filling. Left: unprojected fundamental band. Middle: pseudo DEM with voids. Right: pseudo DEM with void filling.

Step 5: Band alignment

Each band is aligned to the fundamental band using the same orthorectification process of step 2. The newly created pseudo DEM and the unprojected fundamental band are used as auxiliary data during this band alignment. The only difference between the two orthorectification steps is the RPC polynomial order can be lowered for band alignment. Using a DEM during band alignment is desirable as it improves the alignment for bands recorded at different times and/or viewing angles. This effect is evident in data recordings from satellites such as RapidEye [6]. Fig. 7 shows the results for band alignment using a pseudo DEM.


155

Fig. 7: Band alignment using pseudo DEM from step 5. Left: before band alignment. Right: after band alignment.

Step 6: Translation map merge, band alignment, projection and orthorectification

At this stage, the translation maps obtained from band alignment, projection and orthorectification are merged. The resulting merged map can be used to warp unprojected pixels from all bands directly into a band aligned and orthorectified state in only one resampling step. As the projection map ( ) and orthorectification map ( are already merged into (output from step 3), this can be used instead of the individual translation maps. The same merging procedure from step 3 is used here. The output of this step is calculated as follows.

(15)

where:

the merge method from step 3

the band number in question

No merge for the fundamental band is required, can be used directly for the final translation. This is because if there was to be a band alignment translation map for the fundamental band to itself, it would not have resulted in any warping, thus would be stationary.

(16)

where:


156

Step 7: Final product generation

The first and final resample on the unprojected pixels are executed using the combined translation map obtained from step 6. Each band will have its own translation map capable of transforming the unprojected band into a correctly band aligned and orthorectified state. Cubic-interpolation is used as the resampling function.

(17)

(18)

with:

the cubic-interpolation resampler

the band number

Case study using RapidEye data

As a test case, RapidEye data was processed using the methodology describe above. RapidEye images consist of five bands (blue, green, red, red-edge and near-infra-red) [6]. The bands recorded by the RapidEye satellites have different viewing angles and temporal offsets [6]. These offsets normally make it hard to geometrically align and terrain correct the bands without having good attitude and ephemeris data or without a complete sensor model. As this method does not require any attitude and ephemeris data, RapidEye data is ideal for testing purposes.

The red band was chosen as the fundamental band. A SPOT-6 panchromatic mosaic was used as reference image together with the SRTM 1-arc second DEM. The target projection was set to UTM-S zone 34 with a GSD of 10 m per pixel.

In these test cases the band alignment and orthorectification stages use the same RPC polynomials ‘1 x y z xx xy yy’, without any denominators. It should be noted though that none of the steps need to be limited to any specific form of polynomial. The main characteristics of the RapidEye data used for band alignment verification are shown in Table 1.

Acquisition date 16 January 2010 09:50:15 UTC

Radiometric corrected Yes

Geometric correction level None

Atmospheric corrected No

Dimensions X: 11980 pixels

Y: 9468 pixels

Geographic location (description) False Bay, South Africa

Geographic location (latitude, longitude) UL = -33.80058347965510°, 18.01550094998301°

UR = -33.90934682856813°, 18.93789440403745°

LR = -34.45138319890282°, 18.80777391452736°

LL = -34.34208587142159°, 17.87916041433490°

Table 1: The main characteristics of the investigated RapidEye data for band to-band comparison.


157

Band to-band comparisons for all five bands of the resulting band aligned and orthorectified image are shown in Table 2 and Table 3. Table 2 shows the horizontal residuals (between matching feature points) and Table 3 shows the vertical residuals.

Horizontal blue green red red-edge NIR

blue 0,00 (0,00) 0,02 (0,09) 0,00 (0,20) 0,01 (0,34) 0,06 (0,80)

green 0,00 (0.00) 0,02 (0,23) 0,03 (0,29) 0,09 (0,69)

red 0,00 (0,00) 0,06 (0,11) 0,31 (3,16)

red-edge 0,00 (0,00) 0,09 (0,17)

NIR 0,00 (0,00)

Table 2: Horizontal residual band to-band mean (and standard deviation) in pixels (worst value in bold).

Vertical blue green red red-edge NIR

blue 0,00 (0,00) 0,13 (0,15) 0,06 (0,32) 0,11 (0,38) 0,28 (0,75)

green 0,00 (0,00) 0,07 (0,34) 0,09 (0,33) 0,28 (0,76)

red 0,00 (0,00) 0,07 (0,11) 0,25 (0,83)

red-edge 0,00 (0,00) 0,11 (0,16)

NIR 0,00 (0,00)

Table 3: Vertical residual band to-band mean (and standard deviation) in pixels (worst values in bold).

Fig. 8 shows the band-to-band comparison on the band aligned and orthorectified image respectively. The land mass is highly correlated, whereas poor correlation is achieved over the cloudy area in the lower left corner. This poor correlation is caused by the viewing angle offsets between the different bands recorded by RapidEye. This results in spatial offset between pixels of different bands over cloudy areas. The same can be said for off-nadir mountainous areas, but using the pseudo DEM during band alignment alleviates this effect.

Matching feature points are found between an input band and the reference image. Both images are divided into an evenly spaced grid of 66 x 66 pixels. The 33 x 33 pixels around the centre area of each grid block of the input band is extracted as a feature chip. This chip is then matched to an area in the corresponding block in the reference image using a moving window approach. The point in the reference image with the best correlation is deemed the same feature point. The correlation used accounts for subpixel locations [2]. The residuals measure the disparity between these two corresponding feature points.


158

Fig. 8: Band alignment verification. Top-left: fundamental band (RapidEye band 3) after band alignment and orthorectified. Bottom-left: Comparison band (RapidEye band 1). Right: Geometric residual result (green:

error < 0,5, cyan: 0,5 < error < 1, blue: 1 < error < 2, yellow: 2 < error < 5, red > 5 pixels).

A similar comparison is used to verify the orthorectification accuracy. The band aligned and orthorectified fundamental band is compared to the reference image to validate the geolocation accuracies. The main characteristics of the RapidEye data used for orthorectification verification are shown in Table 1 and Table 4.

Acquisition date 16 January 2010 09:50:07 UTC

Radiometric corrected Yes

Geometric correction level None

Atmospheric corrected No

Dimensions X: 11980 pixels

Y: 9468 pixels

Geographic location (description) Yzerfontein to Robben Island (in latitude), South Africa

Geographic location (latitude, longitude) UL = -33.27565473051997°, 18.14667278015216°

UR = -33.38391556114409°, 19.06319551487038°

LR = -33.92635504322634°, 18.93382425765515°

LL = -33.81757516070597°, 18.01123823653822°

Table 4: The main characteristics of the investigated RapidEye data for orthorectification comparison.

Table 5 shows the residuals (between matching feature points) for orthorectification in the horizontal and vertical axes.

Vertical Horizontal

Mean 0,155037 0,321686

StdDev 0,120496 0,273709

Table 5: Orthorectification residuals (pixels) in the horizontal and vertical axes between the fundamental band

of the band aligned and orthorectified image and reference image.


159

Fig. 9 compares the fundamental band to reference image.

Fig. 9: Orthorectification verification. Top-left: fundamental band (RapidEye band 3) after band alignment and

orthorectified. Bottom-left: reference image (SPOT-6). Right: geometric residual result (green: error < 0,5, cyan: 0,5 < error < 1, blue: 1 < error < 2, yellow: 2 < error < 5, red > 5 pixels).

Conclusion

It was shown that a generic sensor model for push broom sensors can be created using multiple RPCs. Each translation map produced by these RPCs can be merged into a single translation map (one per band). During the merging stages, a resampler is used; however, no pixel data is resampled more than once to produce the final dataset. Ephemeris and attitude information are not used during processing, making this method suitable to geometrically correct satellite pixel data with insufficient metadata. Results have shown that processing RapidEye images using this method produces band alignment results well below a residual of one pixel. The geographic accuracy of the orthorectification result was also shown to have residuals below one pixel. These results can be improved further if a RPC model refinement step is introduced, and more advanced void filling algorithm is used. The results shown here did not refine or reject any GCPs found during processing.

Acknowledgements

This work would not have been accomplished without the expertise of the employees at Pinkmatter Solutions and Wolfgang Lück from Forest Sense. Their continued support, constructive feedback and ideas led to many solutions, making this work a reality. All images are subject to copyright by RapidEye.

References

[1] Frank Warmerdam and Gerald Evenden: “PROJ.4 - Cartographic Projections Library,” GitHub repository: https://github.com/OSGeo/proj.4/wiki 2015.

[2] Hassan Foroosh, Josiane Zerubia and Marc Berthod: “Extension of Phase Correlation to Subpixel Registration”, IEEE Transactions on Image Processing, Vol. 11, No. 3, March 2002.

[3] Zhen Xiong and Yun Zhang: “A Generic Method for RPC Refinement Using Ground Control Information”, Photogrammetric Engineering & Remote Sensing, Vol. 75, No. 9, September 2009.

[4] Robert Keys: "Cubic Convolution Interpolation for Digital Image Processing," IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 29, No. 6, pp. 1153 - 1160, Dec 1981.

[5] Donald Shepard: “A Two-Dimensional Interpolation Function for Irregularly-Spaced Data,” Proceedings of the 1968 23rd ACM National Conference, pp. 517-524, 1968.

[6] BlackBridge, “Satellite Imagery Product Specification,” Technical Report, Version 6.1. Available: www.blackbridge.com/rapideye/upload/RE_Product_Specifications_ENG.pdf , April 2015.

Contact Philip Bouwer, Pinkmatter Solutions, Tel 012 998-3492, [email protected]

band alignment and orthorectificatio n of satellite images ... · geomatics indaba proceedings 2015...

Documents