n.1

Multitemporal validation of an unmixing-based MERIS cloud screening algorithm

L. Gomez-Chova1,*, R. Zurita-Milla2, G. Camps-Valls1, L. Guanter3, J. Clevers2, J. Calpe1, M. E. Schaepman2 and J. Moreno3 (1) GPDS, Dept. Ingeniería Electrónica, Universidad de Valencia, Dr Moliner 50, 46100, Burjasot, Spain (2) Centre for Geo-Information (CGI), Wageningen University, P.O. Box 47, 6700 AA Wageningen, The Netherlands (3) LEO, Dept. de Física de la Tierra y Termodinámica, Universidad de Valencia, Dr. Moliner 50, 46100, Burjasot, Spain *E-mail corresponding author: luis.gomez-chova@uv.es ABSTRACT - The operational use of MERIS images can be hampered by the presence of clouds because this instrument works in the visible and near-infrared part of the electromagnetic spectrum. This work presents a cloud screening algorithm that takes advantage of the high spectral and radiometric resolutions of MERIS and the specific location of some of its bands to increase the cloud detection accuracy. In order to validate the proposed algorithm we set up a real multitemporal land cover mapping application over cloudy areas. A temporal series of MERIS FR images acquired over The Netherlands was used to derive sub-pixel land cover composition by means of linear unmixing techniques.

1 INTRODUCTION

Two of the key features of the MERIS/ENVISAT instrument (Rast, 1999) are its temporal resolution (revisit time: 3 days) and its spatial coverage (swath: 1150 km). MERIS also provides data at unprecedented spectral and spatial resolutions: 15 narrow bands and 300 m pixel size in full resolution (FR) mode. Therefore, this instrument has a great potential for multitemporal studies both at regional and at global scales. However, the operational use of MERIS images can be hampered by the presence of clouds because this instrument works in the visible and near-infrared part of the electromagnetic spectrum (Simpson, 1999). In this respect, an automatic and accurate cloud screening method is essential because it will allow the use of partially cloudy images. This will facilitate the elaboration of MERIS products and improve the usability of MERIS time series.

This work presents a cloud screening algorithm that takes advantage of the high spectral and radiometric resolutions of MERIS and the specific location of some of its bands (oxygen and water vapour absorption bands) to increase the cloud detection accuracy (Gomez-Chova, 2006). In order to validate the proposed cloud screening algorithm we set up a real multitemporal land cover mapping application over cloudy areas, where a temporal series of MERIS images is used to derive sub-pixel land

cover composition by means of linear spectral unmixing techniques.

2 STUDY AREA AND DATASETS

The Netherlands was selected as study area because of its frequent cloud coverage, the heterogeneity of its landscapes and the availability of an up-to-date high spatial resolution land use database. 2.1 MERIS FR data

A temporal series of MERIS FR level 1b images acquired over The Netherlands in 2003 was used to illustrate this work. Table 1 shows the selected dates for the multitemporal unmixing as well as the date of the image that was only used to validate the cloud screening algorithm: the 22nd of April (Fig. 1).

Table 1. MERIS acquisition dates 18 February 14 July 16 April 6 August 22 April 15 October 5 May 8 December

2.2 Reference dataset

The latest version of the Dutch land use database, LGN5, was used as a reference in this study. This geographical database has a pixel size of 25m and a detailed legend consisting of 39 classes. The LGN5 is based on multi-temporal classification of high resolution satellite data and the integration of ancillary data (Hazeu, 2005).

n.2

In order to facilitate the unmixing, the LGN5 classes were first thematically aggregated into the nine main land cover types of The Netherlands: grassland, arable land, deciduous forest, coniferous forest, water, built-up, greenhouses, bare soil (including sand dunes), and natural vegetation. Then, the LGN5 was spatially aggregated in order to match the MERIS FR pixel size. To do so, a majority filter with a window of 12 by 12 LGN5 pixels (25 m × 12 = 300 m) was used. During this spatial aggregation process, the abundances of the different land cover types present in the final pixel of 300 m were recorded.

2.3 MERIS pre-processing

MERIS level 1 products are provided in top of the atmosphere (TOA) radiance (radiometrically calibrated data). From this data we computed the MERIS TOA reflectances in order to remove the dependence of the data on particular illumination conditions (day of the year and angular configuration). A second step in the pre-processing of the images was the co-registration. Multitemporal studies require an accurate co-registration so that the correspondence between pixels of different dates is ensured. Nevertheless, a perfect correspondence between pixels of different dates is very difficult to obtain because of differences in observation angles (in our case each MERIS acquisition date belongs to a different ENVISAT orbit) and because of the so-called resampling effects (e.g., Moire patterns). In order to minimise these effects, we computed the “real” land cover abundances as seen by MERIS for each date. This provided us with abundances that can be used to do a fair validation of the unmixing results. The “real” abundances were computed by first projecting each MERIS image into the original 25m grid of the reference dataset. Then, the abundances of the different land cover types present in the area observed by each MERIS CCD element were computed. The class having the highest abundance was also used to produce a land cover classification for each date so that both a sub-pixel and a per-pixel validation of the

results could be done. After this, each MERIS image and its corresponding sub-pixel abundances and land cover map were again reprojected into the same coordinate system as used before but this time with a grid of 300 by 300m (i.e., MERIS nominal pixel size). A nearest neighbour interpolation method was used so that the original values recorded by MERIS were not modified.

3 CLOUD SCREENING

3.1 Cloud screening algorithm

The proposed cloud screening algorithm (Gomez-Chova, 2006) consists of the following steps.

1. Feature extraction: Physically-inspired features are extracted to increase separability of clouds vs. any-other surface type taking into account that the measured spectral signature depends on the illumination (solar and observation conditions: TOA reflectance), the atmosphere (cloud height: oxygen and water vapour atmospheric absorptions), and the surface (cloud reflectance: white and bright).

2. Image clustering: An unsupervised Expectation-Maximization (EM) clustering algorithm is applied using all extracted features in order to obtain all the existing clusters over the scene and a probabilistic membership of pixels to each cluster.

3. Spectral unmixing: In order to obtain a cloud abundance map for every pixel in the image −rather than flags or a binary classification− a spectral unmixing algorithm is applied to the MERIS image using the spectral information. The spectral signatures of the clusters are considered as the representative pixels of the covers present in the scene, and they are used to build the matrix M in the fully constrained linear spectral unmixing algorithm: each pixel-k is modelled as a mixture of the cluster centers in M, pk=M·ak+ε, thus vector ak contains the abundances of the spectral signatures of the clusters.

4. Cluster labelling: The obtained clusters are labelled into geo-physical classes −or at least the clusters corresponding to clouds are identified−. Once all clusters have been related to a class with a geo-physical meaning, it is straightforward to merge all the clusters belonging to a cloud type (cloud-clusters) as follows. In the clustering of the extracted features, the EM algorithm provides posterior probabilities for each cluster-j (Pjk∈[0,1] and ΣPjk=1), thus the probability of being cloud is computed as the sum of the posteriors of the cloud-clusters: Cloud Probabilityk=ΣPjk ∀j classified as cloud. Similarly, the fraction of cloud is computed as the sum of the abundances of the cloud-clusters: Cloud Fractionk=Σajk ∀j classified as cloud.

Figure 1. Pair of MERIS images acquired the 16th (left) and the 22nd (right) of April.

n.3

The final cloud probability map is obtained combining the Cloud Probability and the Cloud Fraction by means of a pixel-by-pixel multiplication. That is, combining two complementary sources of information processed by independent methods: the cloud probability (obtained from the extracted features), which is close to one in cloud-like pixels and close to zero in remaining areas; and the cloud abundance or mixing (obtained from the spectra).

3.2 Multitemporal validation of the cloud screening

As no “ground truth” of existing clouds is available, the performance of the cloud screening is tested with a multitemporal validation approach. In particular, pairs of cloud-free and cloud-covered images (Fig. 1) are used to detect cloud-pixels by identifying pixels with spectral changes between both dates (t1 and t2) higher than a given threshold. Concerning the spectral change, the spectral angle distance (SAD) is used since it is invariant to multiplicative scaling (being less affected by atmospheric and illumination changes):

( ){ }1 2 1 2arccos ,t t t tSAD = ⋅p p p p

(1)

where ⟨·,·⟩ is the dot product operator, and ||·|| is the quadratic norm.

Image pairs are selected to be close in time in order to avoid spectral changes due to temporal evolution of the surface. However, images taken from orbits with three or six days of difference present a significant variation in the viewing geometry. Therefore, pixels with significant changes in composition (>10%) due to the different observation are not considered in the multitemporal cloud screening validation.

4 SPECTRAL UNMIXING

A fully constrained linear spectral unmixing (FCLSU) was applied to each MERIS image (mono-temporal case) as well as to a multitemporal composite (layerstack) of all the MERIS images. The FCLSU, which guarantees a physical interpretation of the results, can be formalized as follows:

( )1

nc

i c cic

p f μ ε=

= ⋅ +∑

Subject to:

∑=

=≤≤nc

ccc fandf

1110

(2)

where pi is the pixel value for the band-i, nc represents the number of classes that are being unmixed, fc is the fraction of class-c present in the pixel, and μci is the pure signal of the class-c in the band-i (this signal is

commonly known as “class endmember”). Finally, the term ε represents the per band residual error.

It is worth noting that, in the case of the monotemporal unmixing, the cloud mask was used to remove from the unmixing all the cloudy pixels. In the case of the multitemporal unmixing, the cloud mask was used to remove the cloudy dates of each pixel so that the unmixing could be done using the maximum available information for each pixel. This implies that the number of dates used to unmix each pixel is not constant throughout the image.

4.1 Selection of endmembers

An enhanced manual endmember selection method was followed in this study because of the availability of the LGN5 database, the class abundances, and cloud product for each MERIS date. First, the abundances of each class were summed up for all dates. Next, these “multitemporal abundances” were used to compute a multitemporal version of the standard purity index, SPI, (Zurita-Milla, 2006):

( ) ( )11

max −−= ∑=

ncffSPInc

cclassc

(3)

where fc represents the total abundance of each class-c in a given pixel and fmaxclass is the maximum abundance.

The multitemporal SPI equals one when a given pixel has only one class for all the dates under study, and it equals zero when the sum of the abundances for all dates results in the same number (1/nc) for all classes. A SPI threshold of 0.90 was used to define “pure multitemporal” areas for all the classes. In order to minimise adjacency or environmental effects an erosion filter of 3 by 3 was subsequently applied to the pure pixels identified with the SPI. This filter was not applied to the greenhouses class since this class is very small. After that, we used the cloud product information to remove the pixels that were identified as cloudy for each date. Finally, the “multitemporal pure” areas were used as a mask over all the MERIS FR images to get the spectral signature of the endmembers.

4.2 Accuracy assessment

The accuracy of the unmixing results was assessed both at a sub- and per-pixel levels. The abundances and classification map computed/produced during the spatial aggregation of the LGN5 were used as ground truth for all images. A fairer comparison could have been obtained by using the abundances that were computed for each date (c.f. section 2.3). Nevertheless, here we decided to use a unique ground truth so that

n.4

all the monotemporal classification results could be compared among them and with the results obtained from the multitemporal unmixing. Recall, however, that the endmembers were selected using the abundances computed for each date.

Assuming that the estimated fractions are correctly positioned within the pixel, a kind of overall sub-pixel accuracy (OSA) can be computed as follows:

{ }cLGN

cc

nc

cc

nc

cc

nc

cc

ffd

dfdOSA

,min 5111

=

== ∑∑∑===

(4)

where dc are the correctly classified abundances for each pixel. These abundances can be computed as the minimum of fc

LGN5 and fc , which respectively are the LGN-based and the unmixed abundances. Notice that the sum of fc for all classes adds to unity (Eq. 2).

After the sub-pixel accuracy assessment, the abundances were used to produce a land cover classification map by selecting for each pixel the class having the highest abundance. The classification accuracy of these maps was subsequently computed by comparing them with the reference map (LGN5 at 300m). The confusion matrix and the kappa index were used for this comparison.

5 RESULTS AND DISCUSSION

5.1 Cloud screening results

Undetected clouds can hamper the selection of endmembers and seriously affect the quality of the unmixing of cloud contaminated pixels. Therefore, an accurate cloud screening is needed to remove all the cloudy pixels from the final analysis.

A hard cloud mask was obtained for each date by applying the same threshold to all the cloud probability maps obtained with the proposed algorithm. As no ground truth −indicating cloudy pixels− was available, in principle we could merely analyze the performance of the proposed method by visual inspection. The analysis of the results showed an excellent cloud screening performance even in thin clouds and cloud borders. The only exception was a small amount of pixels belonging to the classes greenhouses (sun glint on glass roofs) and bare soil (sand dunes). These pixels were identified as clouds because of two reasons: (i) they have similar reflectance behaviour as clouds; and (ii) they represent less than 0.5% of The Netherlands and, therefore, they are not statistically representative in the clustering process of the extracted features (cloudy pixels usually have a lower atmospheric absorption than surface pixels due to their height).

In the case of MERIS, images covering the same area are acquired within few days. Therefore, image pairs can be used to perform a multitemporal validation of the cloud screening (c.f. section 3.2). In this study, we used the pair of images acquired the 16th (cloud free) and the 22nd (cloudy) of April (Fig. 1). Figure 2 shows on the left the multitemporal spectral change computed with Eq.1, and on the right the cloud probability map for the 22nd of April as provided by the proposed cloud screening algorithm.

The cloud screening accuracy was assessed by comparing the hard cloud mask with a “true mask”, which is obtained by applying an empirical threshold to the multitemporal spectral change. Considering the cloud mask as a binary classification, the overall accuracy (91.71 %) and the kappa statistic (0.83) showed the good detection accuracy. However, one has to interpret this assessment as a comparison of the multitemporal and proposed method more than as an absolute accuracy, since the multitemporal approach is not certainly true. Figure 3 shows the comparison of

Figure 2. Left: multitemporal spectral change computed from images acquired the 16th and the 22nd of April. Right: cloud probability mask provided by the proposed cloud screening algorithm for April 22nd.

Temporal vs. Spectral cloud screening

0:Background

1:Land / Land

2:Cloud / Land

3:Land / Cloud

4:Cloud / Cloud

Figure 3. Comparison of the multitemporal cloud flag and the obtained cloud mask (discrepancies are shown in blue when our algorithm detects cloud and in red when pixels are classified as cloud free).

n.5

both cloud masks. On the one hand, when our algorithm detects more cloudy pixels (blue), good agreement with cloud borders can be seen. Therefore, one can assume that the proposed method provides better recognition in cloud borders and thin clouds. On the other hand, differences when our algorithm classify as cloud free are shown in red. One can see that these areas correspond to the boundaries between land cover types, where spectral changes are probably due to the different viewing geometry of the two dates.

5.2 Multitemporal unmixing

Figure 4 shows the spectral signature of the endmembers for the seven dates selected in this study. Grassland presents the highest NIR reflectance all year around. During the months of May, July and August the endmember of deciduous forest also shows high reflectance (high greening of vegetation). The rest of the vegetated classes appear to have a very similar spectral signature. High confusion is, therefore, expected among these classes.

The unmixed abundances for each date and the multitemporal approach were compared with the abundances computed during the spatial aggregation of the LGN5 database. The overall sub-pixel accuracy (OSA; Eq. 4) was used to do this comparison. Then, the classification obtained from the abundances (Fig.5) was used to compute the overall classification accuracy (OA) and the kappa statistic. Table 2 summarises the results. Three items should be noticed from this table:

• As expected, the multitemporal approach yielded the highest classification results, since adding the temporal evolution (phenology) simplifies the discrimination of spectrally similar land cover types.

• The difference between the classification results of the best monotemporal image (April) and the multitemporal approach is not very large. Errors in the multitemporal case can be produced by the within class heterogeneity (land covers with

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

0.1

0.2

0.3

0.4

0.5

TOA

Refle

ctan

ce

GrasslandArable landGreenhousesDeciduous forestConiferous forestWaterBuilt−upBare soilNatural vegetation

Figure 4. Pure spatial and multitemporal endmembers selected from the multitemporal dataset for each class.

0:No data

1:Grassland

2:Arable land

3:Greenhouses

4:Deciduous forest

5:Coniferous forest

6:Water

7:Built up

8:Bare soil

9:Natural vegetation

Figure 5. Left: Classification obtained from the FCLU of the multitemporal series. Right: LGN5 resampled to 9 classes and 300 m used as ground truth.

n.6

different phenology mixed in one class –e.g. arable land class).

• The OSA and the OA values are in the same order of magnitude. However, the OSA refers to the sub-pixel abundances and therefore it inherently contains more information than the classification.

Finally, Table 3 shows the user’s and producer’s accuracies for the multitemporal case. Notice that the classes greenhouses, deciduous forest, bare soil and natural vegetation present the poorest producer’s and user’s accuracies due to the aforementioned reasons.

6 CONCLUSIONS

This work has presented a cloud screening algorithm that correctly identifies the location and abundance of clouds in MERIS images. The algorithm has been validated against a cloud mask obtained with a multitemporal change detection approach. Despite the fact that the proposed method only uses the information of the “cloudy image”, results show that our method offers a better discrimination of thin clouds and clouds borders. This accurate cloud screening algorithm enables a more efficient use of MERIS images.

This study has also shown that the use of MERIS FR data has a great potential for sub-pixel land cover mapping. The unmixing of MERIS FR time series performed better than the unmixing of single dates. Two refinements might still improve the classification accuracy: refining the land cover classes to reduce within class heterogeneity (e.g., split the arable land class); and the selection of dates for the unmixing (vegetation phenophases)

ACKNOWLEDGMENTS

This paper has been partially supported by the Spanish Ministry for Education and Science under project

DATASAT ESP2005-07724-C05-03, and by the Generalitat Valenciana under project HYPERCLASS /GV05/011. The contribution of R. Zurita Milla is granted through the Dutch SRON GO programme (EO-061).

REFERENCES

Gomez-Chova, L., et al., 2006, New Cloud Detection Algorithm for Multispectral and Hyperspectral Images: Application to ENVISAT / MERIS and PROBA / CHRIS Sensors. IEEE International Geoscience and Remote Sensing Symposium 2006. Denver, Colorado, July 31-August 4, 2006.

Gomez-Chova, L., et al., 2006, Cloud detection for MERIS multispectral images. Proceedings of the MERIS (A)ATSR Workshop. ESA Publications Division, ESA SP-597.

Hazeu, G., 2005, The Dutch Land Use Database LGN. [web page] http://www.lgn.nl/

Rast, M., and Bezy, J.L., 1999, The ESA Medium Resolution Imaging Spectrometer MERIS a review of the instrument and its mission, International Journal of Remote Sensing, Volume 20, Issue 9, Jun 1999, Pages 1681 – 1702.

Simpson, J., 1999, Improved cloud detection and cross-calibration of ATSR, MODIS and MERIS data. In ESA-SP-479, ATSR International Workshop on the Applications of the ERS along track scanning radiometer, ESRIN, Frascati, Italy, June, 1999.

Zurita-Milla R., Clevers, J. G. P. W., Schaepman, M. E. and Kneubuchler, M., 2006, Effects of MERIS L1b radiometric calibration on regional land cover mapping and land products. International Journal of Remote Sensing (In press).

Table 2. Performance of the land cover classification using the FCLU at both subpixel (Overall Subpixel Accuracy, OSA) and per pixel (Overall Accuracy, and Kappa statistics) scales.

Data Feb Apr May Jul Aug Oct Dec Multitemporal OSA 44.77 57.44 47.71 54.52 46.97 50.18 47.19 59.23

Kappa 0.36 0.49 0.39 0.45 0.36 0.37 0.35 0.52 OA 46.44 58.58 50.33 56.10 46.81 49.88 47.55 62.29

Table 3. Producer’s Accuracy (PA) and User’s Accuracy (UA) of the multitemporal unmixing classification.

Classes Grass-land

Arable land

Green-house

Decid. Forest

Conif. Forest Water Built-

up Bare soil

Natural Veget.

PA [%] 72.35 70.17 34.28 10.32 44.75 89.92 65.15 15.61 12.60

UA [%] 70.82 44.17 39.93 39.35 71.54 85.79 42.35 38.92 25.39