AIRS Radiance and Geophysical Products:

Methodology and Validation

Mitch Goldberg , Larry McMillin

NOAA/NESDIS

Walter Wolf, Lihang Zhou, Yanni Qu and M. Divakarla

Science ActivitiesScience Activities

Data compression.Data compression. Validate and improve radiative transfer calculations.Validate and improve radiative transfer calculations. Cloud detection and clearing.Cloud detection and clearing. Cloud productsCloud products Channel selection (super channels).Channel selection (super channels). Validate and improve retrieval algorithms.Validate and improve retrieval algorithms. Trace gasesTrace gases Surface emissivitySurface emissivity Use MODIS to improve AIRS cloud detection and cloud Use MODIS to improve AIRS cloud detection and cloud

clearingclearing Radiance bias adjustmentsRadiance bias adjustments Forecast impact studiesForecast impact studies

TOPICS

Use of principal components (a.k.a. eigenvectors) for data compression.

Surface emissivity

Cloud detection

AIRS Geophysical Products

Microwave-only retrieval of sfc emissivity, sfc temperature, sfc type and profiles of temperature, water vapor and cloud liquid water.

AIRS retrieval of cloud amount and height, sfc emissivity, sfc temperature, and profiles of temperature, water vapor and ozone.

AIRS has two retrieval steps – very fast eigenvector regression followed by a physical retrieval algorithm.

Data Compression Data Compression Advanced IR sounder data are very large compared with current Advanced IR sounder data are very large compared with current

sounders (1 orbit ~ 2GB vs. 8 MB) Much larger for GIFTS.sounders (1 orbit ~ 2GB vs. 8 MB) Much larger for GIFTS.

Information is not independent. Principal component analysis Information is not independent. Principal component analysis (PCA) is often used to reduce data vectors with many (PCA) is often used to reduce data vectors with many components to a different set of data vectors with much fewer components to a different set of data vectors with much fewer components that still retains most of the variability and components that still retains most of the variability and information of the original data information of the original data

Data are rotated onto a new set of axes, such that the first few Data are rotated onto a new set of axes, such that the first few axes have the most explained variance.axes have the most explained variance.

Principal component scores are provided instead of the Principal component scores are provided instead of the individual channels.individual channels.

Individual channels can be reconstructed with minimal signal Individual channels can be reconstructed with minimal signal loss with added benefit of noise reduction.loss with added benefit of noise reduction.

Generating AIRS eigenvectors

Collect an ensemble of AIRS spectra (2378 channels).

The radiances are normalized by expected instrumental noise (signal to noise)

Compute the covariance matrix S

Compute the eigenvectors E and eigenvalues S = E ET

E = matrix of orthonormal eigenvectors (2378x2378) = vector of eigenvalues (explained variance)

Training Ensemble

Eigenvectors are generated from a spatial subset of AIRS data (200 mbytes vs 30 GB full data)

Eigenvectors are generated daily. A static set of eigenvectors is used, but the

ensemble is occasionally updated with new structures.

When the ensemble is updated a new set of eigenvectors is also updated.

Locations used in generating eigenvectors

Applying AIRS eigenvectors

On independent data – compute principal component scores.

P = ET R ; elements of R = (ri- ri ) /ni

Invert equation and compute reconstructed radiances R*.

R* = E P

Reconstructed radiances are used for quality control.

Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2

i = 1 ….N channels

1 7497.60 2 1670.40 3 945.52 4 496.01 5 284.01 6 266.30 7 156.95 8 139.67 9 88.27 10 72.83 11 60.03 12 53.42 13 45.01 14 39.72 15 34.54 16 26.57 17 22.62 18 17.60

19 14.68 20 13.49 21 12.28 22 11.32 23 10.70 24 9.08 25 8.24 26 7.85 27 6.77 28 5.98 29 5.83 30 5.39 31 5.34 32 4.98 33 4.34 34 4.09 35 3.62 36 3.48

37 3.38 38 3.11 39 2.82 40 2.53 41 2.41 42 2.39 43 2.34 44 2.24 45 2.03 46 1.86 47 1.78 48 1.71 49 1.65 50 1.61 51 1.54 52 1.52 53 1.35 54 1.34

55 1.25 56 1.19 57 1.16 58 1.15 59 1.09 60 1.05 61 1.02 62 0.98 63 0.90 64 0.86 65 0.81 66 0.80 67 0.78 68 0.77 69 0.73 70 0.72 71 0.70 72 0.66

Square root of the eigenvalues

Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2

i = 1 ….N channels

Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2

i = 1 ….N channels

Monitoring Eigenvectors

Monitoring eigenvectors is critical

Eigenvectors may need to be updated due to new structures that were not in the original ensemble

12/4/00 reconstruction scores

Monitoring reconstruction score is important

Days

July Aug Sep Oct Nov Dec Jan Feb

Noise

Noise free 75 PCS

Observed vs noise-free reconstructed vs noise-free.Observed vs noise-free reconstructed vs noise-free.

Noise ReductionNoise Reduction

“ Observed” Reconstructed

Observed vs. ReconstructedObserved vs. Reconstructed

New Plan

Generate full spatial resolution AIRS principal component score datasets

Size ~ 5 MB instead of 150 MB per six minute granule

Surface emissivity

Retrieval error based on 18 channels

Background Std dev.

Retrieval error

Clear detection

BACKGROUND

NWP centers will assimilate clear radiances

Need very good cloud detection algorithm

Very important for radiance validation and to initiate the testing of the level 2 retrieval code.

Cloud Detection over Ocean Use VIS/NIR channels during day.

Compare SST with 2616 cm-1 at Night.

Predicting SST from 11 and 8 micron channels (works for day and night)

Predict 2616 from 8 micron channels (night)

11 micron window > 270 K

ONLY 0.5% residual clouds

Cloud detection – Non Sea

Predict AIRS channel at 2390.9 cm-1 from AMSU

FOV is labeled “mostly clear” if predicted AIRS – observed AIRS < 2

AND IF

SW LW IR window test is successful:

[ch(2558.224)-CH(900.562)] < 10 K

Variability of 2390.910 radiance within 3x3 < 0.0026

Clear Detected Fovs Cloud cleared casesClear Detected Fovs Cloud cleared cases

Future Work – Merge MODIS and AIRS

High spatial resolution will improve determination of clear AIRS fovs.

High spatial resolution will greatly improve clear estimate needed for cloud clearing.

MODIS Sounder Radiance Product

MODIS has HIRS-like sounder channels – but at high spatial resolution (1 km).

Find a few clear MODIS fovs in a 50 x 50 km area should provide a yield of 80% -- similar to AMSU

Summary Busy getting ready for real AIRS data Simulating AIRS in real-time has provided a means to develop ,

test and validate the delivery of products to NWP centers, AND created a platform to develop scientific tools to analyze the

data and test algorithms. Early releases of the data should be available 3 months after launch Final radiance products ~ 7 months Retrievals ~ 12 months First activity will be to examine biases between measured and

computed radiances and validation of the clear detection algorithm. “Day-2” Utilize MODIS