S. Mackie

(1), C. Merchant

(2), P. Francis

(3)

(1) IAES, University of Edinburgh ,West Mains Rd, Edinburgh EH9 3JN, U.K., Email:s.l.Mackie@sms.ed.ac.uk

(2) IAES, University of Edinburgh ,West Mains Rd, Edinburgh EH9 3JN, U.K., Email:c.merchant@ed.ac.uk

(3) NWP Division, Met. Office, Fitzroy Road, Exeter EX1 3PB, U.K., Email: pete.francis@metoffice.gov.uk

ABSTRACT

Cloud detection always relies on some knowledge of

how clear and cloudy observations will differ. In a full

Bayesian determination of the probability that an

infrared image pixel contains cloud, an estimate of the

brightness temperature distribution for clear and cloudy

cases is required. A method for estimating this

distribution for cloudy atmospheric states through

exploitation of the knowledge already held about an

imaged scene is presented here. Relationships are found

between cloud properties and the brightness-

temperature predictions of a fast radiative transfer

model, run with atmospheric information specific to the

imaged scene. This means that the number of model

runs can be limited, without limiting the number of

clouds represented in the distribution. The technique is

demonstrated here in a case study, the results of which

suggest that clear areas of an image can be identified

with more certainty.

1. INTRODUCTION

As a global source of information on the Earth’s

atmospheric state, satellite data require accurate

interpretation. Interpretation of visible and infrared

imagery usually relies on detection of clouds within an

imaged scene. Standard operational detection is

performed on imagery by threshold testing, which

produces a mask of clear and cloudy pixels, e.g. [1-4],

following the approach of [5]. The threshold tests are

typically set according to the observations of an expert

through inspection of a number of imaged scenes, which

has led the approach to be criticised as ‘non-

transparent’, and dependent on the retaining of expertise

which may be lost. In addition to these criticisms, [6, 7]

point out that the binary mask produced by threshold

testing methods leaves no flexibility for different

tolerances to cloud contamination – more severe

thresholds that detect more cloud are likely to detect

fewer clear pixels, and vice-versa, although the severity

of the tests used in creating the end-product is often

unknown. It is further pointed out that a climatology

may be used to set the thresholds, but numerical weather

prediction (NWP) fields contain more temporally- and

spatially-specific information which could be exploited

to aid the detection [6, 7].

A semi-probabilistic, physically based alternative to the

threshold-method has been developed [7]. NWP fields

are used to calculate a probability density function

(PDF) of observations corresponding to a clear scene.

The PDF-element corresponding to a recorded pixel

observation is the prior probability of that pixel being

clear, according to the NWP fields. This is combined

with the prior probability of imaging a clear scene at the

pixel location (taken from cloud statistics generated by,

e.g. [8]) to calculate the posterior probability of clear for

the pixel. To make the technique fully-probabilistic and

physically robust, an NWP-conditional PDF for

observations of a cloudy scene is also needed, see Eq. 1.

( )

( )

1

1

,cb xo y P c P

c,b

xo

y P c P b

,xo

yc P

−

����

�

�

����

�

�

��

�

�

��

�

�

+=��

�

� (1)

( )cP , ( )cP are the prior probabilities of clear, c, and

not clear, c ; ��

���

�

�� c , x y, P ,c x yP

bobo are the

PDFs probabilities of the observations yo, given the

NWP fields xb, and clear or cloudy conditions.

The clear PDF is generated by running a radiative

transfer model (RTM) on the NWP-profile information,

and accounting for sensor noise and profile uncertainties

using Gaussian error assumptions. At present, [7] uses

an empirical distribution of observations recorded for

cloudy scenes in place of an NWP-conditional PDF for

cloudy scenes because of the difficulty in carrying out

such forward-modelling for cloudy scenes.

Rather than forward-modelling one set of atmospheric

conditions, as for the clear-sky case, the PDF must

represent observations for a range of atmospheric states,

with clouds at different altitudes, with different optical

depths and filling different fractions of the pixel. To be

useful operationally, the PDF must be generated

quickly, making it impractical to cover the range of

possible cloudy conditions with individual RTM runs.

The following sections present a method of generating a

PDF for cloudy atmospheric states.

FAST FORWARD MODELLING OF CLOUDY ATMOSPHERIC STATES

_____________________________________________________

Proc. ‘Envisat Symposium 2007’, Montreux, Switzerland 23–27 April 2007 (ESA SP-636, July 2007)

2. METHOD

The NWP profile is forward modelled using RTTOV-7

with 60 altitude levels. Single phase clouds are added to

each of the modelled altitudes in separate model-runs,

with cloud pixel-coverage varying from 10% to 100%,

and cloud ice- or liquid- water path, cwp, varying from

0 to 0.1 kg m-2

. The forward-modelled brightness

temperatures, BTs, are plotted against cwp for each

modelled altitude and cloud fraction and an exponential

curve fitted, e.g. Fig, 1.

Figure 1. Forward-modelled BTs plotted against cwp.

Dia: forward-modelled BTs, line: fitted curve.

The equation for the curve has the form BT = a + b*(1 –

ecwp/c

) , where a, b and c are fitting parameters. With the

curve defined, BTs can be read for clouds with cwp

values other than those modelled, so reducing the

number of RTM necessary runs.

The steepness and minimum BT value of the curve

changes with altitude and pixel coverage as lower pixel

coverage requires a thicker cloud for an optically

saturated observation, and higher clouds will optically

saturate at lower BTs. The parameters are interpolated

to give the fitting parameters for BT-cwp curves at

altitudes and pixel coverages between those that are

forward-modelled. In this way, BTs for clouds with tops

at 10m intervals through the atmosphere, and with pixel

coverages varying from 10%-100% in 1% increments,

can be found without being explicitly modelled.

2.1. Weighting the Clouds

The clouds that contribute to the PDF must be realistic

given the NWP profile. The dataset from [9] was used

to ensure clouds represented in the PDF have cwp of

less than or equal to the maximum cwp recorded at that

altitude. The contribution of each cloud to the final PDF

needs to be weighted by its relative likelihood. A weight

is given to optically thick clouds using the ratio of

clouds in this dataset with cwp greater than or equal to

the maximum modelled cwp, which was chosen to be

beyond the optical saturation point.

An empirical dataset of AATSSR-acquired imagery,

consisting of measurements from January, April, July

and October, was used to calculate a latitude- and

season-specific ratio of cloud-filled pixels to cloud-edge

pixels, e.g. Fig. 2. The AATSR cloud mask was used to

define a pixel as cloud-edge if it was classed as cloud

but had � 1 neighbours classed as clear, or if it was clear

bur had � 1 neighbours classed as cloud. The 8

immediate neighbours of a pixel were used for this, and

cloud pixels for which all 8 neighbours were also

classed as cloud, were defined as cloud-filled. This ratio

was used to weight clouds with 100% pixel coverage

more heavily in the PDF.

Figure 2. Ratio of cloud-filled to cloud-edge pixels for

autumn AATSR dataset

Ice phase clouds are only represented in the PDF at

altitudes for which the temperature (interpolated from

the NWP profile temperature) is below 273.15K. Liquid

phase clouds at temperatures below 273.15K are

represented with a linearly decreasing weight, which

reaches 0 at 233.15K to account for super-cooled water

clouds.

The aim of the work is to enhance the technique

developed by [7] to aid sea surface temperature (SST)

retrieval, and so only clouds which introduce a bias of

0.2K or greater to the retrieved SST are represented in

the PDF.

The distribution of BTs predicted for a cloudy

atmosphere is convolved with Gaussian error

assumptions in the same way as for the clear sky, and

so the PDF is created.

3. USE OF THE CLOUDY-SKY PDF

To assess the performance of the PDF, it can be

included in the full cloud detection code [7] as a direct

replacement for the global distribution currently used.

That is, the PDF generated from one NWP-profile can

be applied to observations from an imaged scene, which

spans more than one NWP grid cell. Some indication of

the profile-dependence of the PDF is given by its

varying performance in areas close to, and far from,

those described by the NWP-profile.

4. RESULTS

The PDF generated for cloudy atmospheric states using

an NWP-profile for an area off the coast of Korea in

May 2005 is shown in Fig.3. This can be compared to

the global distribution of cloudy atmosphere

observations currently used by [7] in Fig. 5. The PDF in

Fig. 3 was generated for a profile centred to the area

marked in yellow on the visible imagery, Fig. 4a. The

spectral probability of clear calculated for this image

using the global distribution is shown in Fig. 4b. The

PDF in Fig. 3, and a PDF from a profile centred on the

cyan box, were each applied to the whole image,

creating the spectral-probability-of-clear plots shown in

Fig. 4c,d.

Figure 3. PDF plotted in 2-dimensions;,filled contours

at 20 equally spaced intervals, spanning range of

distribution (0 –0.0126874). Black contours plotted on a

logarithmic scale, filled contours equally spaced.

Profile taken from yellow (northern) box in Fig. 4a

a b

c d

Probability of Clear

0 0.5 1

Figure 4 a. 1.6m image with marked regions centred on

location of NWP profiles used to generate PDFs; b. Spectral probability of clear calculated for whole image

using global distribution in place of a PDF; c. using

PDF from profile centred on yellow (northern) box; d.

using PDF from profile centred on cyan (southern) box.

Figure 5. The global distribution of cloudy-atmosphere

observations used to create Fig. 4b (black contours on

logarithmic scale).

Some quantitative comparisons were made between the

results based on the global distribution, and those based

on the PDFs from the 2 profiles. A region of 100-pixels2

centred on the profile-centre was considered for the

comparison, shown in table 1 and Fig. 6. Pixels with a

calculated probability of clear greater than 50% were

deemed clear for the comparison, otherwise they were

considered cloud-pixels. Of each of the two classes,

clear and cloud, the number of pixels that the two

technique classed with higher certainty are compared.

Table 1. Comparison of the calculated probability of

clear for pixels in region centred on NWP-profile

location, using the global distribution and using the

NWP-conditional PDF.

profile 1 region (Fig. 4a yellow box)

global

distribution

NWP-conditional-PDF

cld pix 3906 3945

clr pix 36495 36456

% of clr px > 85% 26.9% 56.3%

% of clr px > 90% 0% 37.5%

% cld px < 15% 70.5% 71.9%

% cld px < 10% 66.6% 68.0%

profile 2 region (Fig. 4a cyan box)

global

distribution

NWP-conditional-PDF

cld pix 7476 7295

clr pix 32925 33106

% of clr px > 85% 20.7% 60.4%

% of clr px > 90% 1.5% 44.8%

% cld px < 15% 12.9% 12.1%

% cld px < 10% 11.9% 11.2%

Figure 6 .Histogram of probability values calculated for

pixels using global distribution (dashed line) and

conditional PDF (solid line). Top: region around

profile 1 (yellow box in Fig. 4a); Bottom: region around

profile 2 (cyan box in Fig. 4a)

5. DISCUSSION

The PDF appears to perform a more polarized

classification in the region from which the profiles were

taken. It is expected that this will lead to a reduction in

the false alarm rate.

It is not expected that the PDF out-perform the global

distribution of BTs for cloudy atmospheric states in

regions away from the profile location, and so these

preliminary results show the location-specific nature of

an NWP-profile-dependent PDF. It is intended that the

cloudy PDF eventually be included in the cloud

detection code [7], in the same way as the clear-sky

PDF is at present. That is, it will be calculated for every

available profile within an imaged scene and the results

interpolated between profile-centre locations. The plots

show the results using scene-specific PDFs to be more

polarized in the region where they apply. Classification

of pixels into ‘clear’ and ‘cloudy’ classes can therefore

be done with more certainty.

The PDF could be made more conditional on the NWP

profile, for example by limiting the clouds represented

to those realistic for the profile. It is intended to

investigate this, but it is also anticipated that such

conditions may lead to problems when atmospheric

conditions vary within a profile grid cell, e.g. in the case

of ocean fronts.

These preliminary results are encouraging, showing the

benefits of exploiting scene-specific information to form

a PDF for cloudy pixels in a probabilistic cloud

classification scheme.

6. REFERENCES

1. Stowe, L.L., P.A. Davis, and E.P. McClain,

Scientific basis and initial evaluation of the CLAVR-

1 Global Clear/Cloud Classification Algorithm for

the Advanced Very High Resolution Radiometer.

Journal of Atmospheric and Oceanic Technology,

1999. 16: p. 656-681.

2. Kriebel, K.T., et al., The cloud analysis tool

APOLLO: Improvements and validations.

International Journal of Remote Sensing, 2003.

24(12): p. 2389-2408.

3. Hutchison, K.D., et al., Automated cloud detection

and classification of data collected by the Visible

Infra-red Imager Radiometer Suite (VIIRS).

International Journal of Remote Sensing, 2005.

26(21): p. 4681-4706.

4. Derrien, M. and H. Le Gléau, MSG/SEVIRI cloud

mask and type from SAFNWC. International Journal

of Remote Sensing, 2005. 26(21): p. 4707-4732.

5. Saunders, R. and K. Kriebel, An improved method

for detecting clear sky and cloudy radiances from

AVHRR data. International Journal of Remote

Sensing, 1988. 9(1): p. 123-150.

6. Merchant, C.J., et al., Probabilistic physically-based

cloud-screening of satellite infra-red imagery for

operational sea surface temperature retrieval.

Quarterly Journal - Royal Meteorological Society,

2005. 131(611): p. 2735-2756.

7. Old, C., et al., Generalised Bayesian cloud detection

for thermal and reflectance imagery: software

development and validation. IEEE Transactions on

Geoscience and Remote Sensing, 2007.

8. Rossow, W.B. and R.A. Schiffer, ISCCP cloud data

products. Bulletin, American Meteorological

Society, 1991. 72(1): p. 2-20.

9. Chevallier, F., Sampled databases of 60-level

atmospheric profiles from the ECMWF analyses, in

SAF Programme, E. ECMWF, Editor. 2001. p.

Research Report no. 4.